Refactoring and cleanup (#15)

* rename from docker to development

* refator a bit the acl manager and try to reduce the memory footprint

* update commands

* update readme

* update gitignore

* remove old reference

* update from localhost to vault to match the containername

* update tests

* run tests in parallel

* adds missing tests

* adds extra tests

* more tests

* tidying a bit the code to simplify testing syncACLS

* try to simplify the test a bit before adding new cases

* adds missing tests

.gitignore

@@ -18,5 +18,8 @@
 # vendor/
 .idea/
 
+# .DS_Store
+.DS_Store
+
 # ignore local buid
 bedel

README.md

@@ -1,28 +1,50 @@
 # Bedel 
 
-[![Go](https://github.com/ncode/port53/actions/workflows/go.yml/badge.svg)](https://github.com/ncode/port53/actions/workflows/go.yml)
+[![Go](https://github.com/ncode/bedel/actions/workflows/go.yml/badge.svg)](https://github.com/ncode/bedel/actions/workflows/go.yml)
 [![Go Report Card](https://goreportcard.com/badge/github.com/ncode/bedel)](https://goreportcard.com/report/github.com/ncode/bedel)
 [![codecov](https://codecov.io/gh/ncode/bedel/graph/badge.svg?token=N98KAO33K5)](https://codecov.io/gh/ncode/bedel)
 [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 
-`bedel` is a tool designed to address a specific challenge with Redis: synchronizing users generated outside of the configuration file, such as through the Vault database backend. This utility ensures that Redis acls are up-to-date and consistent across all nodes. More info [here](https://github.com/redis/redis/issues/7988).
+`bedel` is a utility designed to synchronize ACLs across multiple nodes in Redis and Redis-compatible databases like Valkey. It specifically addresses the challenge of managing users created outside the traditional configuration file, such as those generated through the [Vault database backend](https://www.vaultproject.io/docs/secrets/databases/redis). By keeping ACLs consistent across all nodes, Bedel ensures seamless user management and enhanced security in distributed environments. For more information on the underlying issue with Redis, see [Redis Issue #7988](https://github.com/redis/redis/issues/7988).
+
+## Table of Contents
+
+- [Features](#features)
+- [Getting Started](#getting-started)
+  - [Prerequisites](#prerequisites)
+  - [Installation](#installation)
+  - [Running the Tests](#running-the-tests)
+- [Development Setup](#development-setup)
+  - [Configuration](#configuration)
+- [Usage](#usage)
+- [Contributing](#contributing)
+- [License](#license)
 
 ## Features
 
-- Synchronize Redis users created outside the traditional config file.
-- Integration with Vault database backend for user management.
-- Automated and consistent user synchronization.
-- Easy to deploy and integrate within existing Redis setups.
+
+- Automated User Synchronization: Automatically synchronizes Redis users and ACLs across all nodes to maintain consistency.
+- Vault Integration: Seamlessly integrates with HashiCorp Vault's database backend for dynamic user management.
+- Configurable Sync Intervals: Allows customization of synchronization intervals to suit your deployment needs.
+- Lightweight and Efficient: Designed to have minimal impact on performance, even with thousands of users.
+- Easy Deployment: Simple to deploy with Docker Compose or as a standalone binary.
+- Robust Logging: Provides detailed logs for monitoring and troubleshooting.
 
 ## Getting Started
 
 These instructions will guide you through getting a copy of `bedel` up and running on your system for development and testing purposes.
 
 ### Prerequisites
 
-- Redis server setup.
+
+For users:
+- Redis server setup
 - Access to Vault database backend (if using Vault for user generation).
-- Go environment for development.
+
+For developers:
+- Go 1.21 or higher
+- Docker and Docker Compose (for development and testing).
+- Git (for cloning the repository).
 
 ### Installing
 
@@ -35,11 +57,92 @@ $ cd bedel
 $ go build
 ```
 
+2. Go install:
+```bash
+4 go install github.com/ncode/bedel/cmd/bedel@latest
+```
+
 ### Running the Tests
 
 To run the automated tests for this system, use the following command:
 
 ```bash
 $ go test ./...
 ```
-
+
+## Development Setup
+
+Bedel comes with a development environment setup using Docker Compose. This setup includes:
+
+- Three Redis instances (redis0001, redis0002, redis0003)
+- Three Bedel instances (bedel_redis0001, bedel_redis0002, bedel_redis0003)
+- A Vault instance for managing secrets
+
+To start the development environment:
+
+1. Ensure you have Docker and Docker Compose installed.
+2. Navigate to the project root directory.
+3. Run the following command:
+
+```bash
+$ cd config/development
+$ make
+```
+
+This will start all the services defined in the `docker-compose.yaml` file.
+
+### Configuration
+
+The `docker-compose.yaml` file contains the configuration for all services. Here are some key points:
+
+- Redis instances are configured with custom configuration files located in the `./redis` directory.
+- Bedel instances are configured to connect to their respective Redis instances.
+- The Vault instance is set up with a root token "root" and listens on port 8200.
+
+## Usage
+
+Bedel can be run in two modes:
+
+### 1. Run Once Mode
+
+Performs a single synchronization of ACLs from the primary Redis node to the replica.
+
+```bash
+$ bedel runOnce -a <redis-address> -p <password> -u <username>
+```
+
+### 2. Continuous Loop:
+
+Continuously synchronizes ACLs at a defined interval.
+
+```bash
+$ bedel run -a <redis-address> -p <password> -u <username> --sync-interval <duration>
+```
+
+For more options and commands, run:
+```bash
+$ bedel --help
+```
+
+### Configuration file
+
+Bedel can also read configurations from a YAML file (default: $HOME/.bedel.yaml). Command-line options override configurations in the file.
+
+Example Configuration File (~/.bedel.yaml):
+```yaml
+address: localhost:6379
+password: mypassword
+username: default
+syncInterval: 10s
+logLevel: INFO
+aclfile: false
+```
+
+
+## Contributing
+
+Contributions are welcome!
+
+## License
+
+This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

cmd/root.go

@@ -20,6 +20,7 @@ import (
 	"log/slog"
 	"os"
 	"path"
+	"time"
 
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
@@ -63,7 +64,7 @@ func Execute() {
 func init() {
 	cobra.OnInitialize(initConfig)
 	rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.bedel.yaml)")
-	rootCmd.PersistentFlags().StringP("address", "a", "", "address of the slave to manage instance eg: 127.0.0.1:6379")
+	rootCmd.PersistentFlags().StringP("address", "a", "", "address of the instance to manage, e.g., 127.0.0.1:6379")
 	viper.BindPFlag("address", rootCmd.PersistentFlags().Lookup("address"))
 	rootCmd.PersistentFlags().StringP("password", "p", "", "password to manage acls")
 	viper.BindPFlag("password", rootCmd.PersistentFlags().Lookup("password"))
@@ -73,6 +74,8 @@ func init() {
 	viper.BindPFlag("logLevel", rootCmd.PersistentFlags().Lookup("logLevel"))
 	rootCmd.PersistentFlags().Bool("aclfile", false, "defined if we should use the aclfile to sync acls")
 	viper.BindPFlag("aclfile", rootCmd.PersistentFlags().Lookup("aclfile"))
+	rootCmd.PersistentFlags().Duration("syncInterval", 10*time.Second, "interval between sync operations")
+	viper.BindPFlag("syncInterval", rootCmd.PersistentFlags().Lookup("syncInterval"))
 }
 
 // initConfig reads in config file and ENV variables if set.
@@ -91,7 +94,6 @@ func initConfig() {
 		viper.SetConfigName(".bedel")
 	}
 
-	viper.SetDefault("syncInterval", 10)
 	viper.SetDefault("username", "default")
 	viper.AutomaticEnv()
 
@@ -100,12 +102,12 @@ func initConfig() {
 	}
 
 	if !viper.IsSet("address") {
-		fmt.Fprintln(os.Stderr, "address is required")
+		logger.Error("Address is required")
 		os.Exit(1)
 	}
 
 	if !viper.IsSet("password") {
-		fmt.Fprintln(os.Stderr, "password is required")
+		logger.Error("password is required")
 		os.Exit(1)
 	}
 

cmd/run.go

@@ -16,6 +16,11 @@ limitations under the License.
 package cmd
 
 import (
+	"context"
+	"os"
+	"os/signal"
+	"syscall"
+
 	"github.com/ncode/bedel/pkg/aclmanager"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
@@ -27,11 +32,26 @@ var runCmd = &cobra.Command{
 	Short: "Run the acl manager in mood loop, it will sync the follower with the primary",
 	Run: func(cmd *cobra.Command, args []string) {
 		mgr := aclmanager.New(viper.GetString("address"), viper.GetString("username"), viper.GetString("password"), viper.GetBool("aclfile"))
-		ctx := cmd.Context()
-		err := mgr.Loop(ctx)
+		defer mgr.Close()
+		ctx, cancel := context.WithCancel(context.Background())
+		defer cancel()
+
+		// Set up signal handling
+		c := make(chan os.Signal, 1)
+		signal.Notify(c, os.Interrupt, syscall.SIGTERM)
+		go func() {
+			<-c
+			cancel()
+		}()
+
+		syncInterval := viper.GetDuration("syncInterval")
+		logger.Info("Starting ACL manager loop")
+		err := mgr.Loop(ctx, syncInterval)
 		if err != nil {
-			panic(err)
+			logger.Error("Error running ACL manager loop", "error", err)
+			os.Exit(1)
 		}
+		logger.Info("ACL manager loop terminated")
 	},
 }
 

cmd/runOnce.go

@@ -16,12 +16,12 @@ limitations under the License.
 package cmd
 
 import (
-	"github.com/ncode/bedel/pkg/aclmanager"
-	"github.com/spf13/viper"
 	"log/slog"
 	"os"
 
+	"github.com/ncode/bedel/pkg/aclmanager"
 	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
 )
 
 // runOnceCmd represents the runOnce command
@@ -30,22 +30,23 @@ var runOnceCmd = &cobra.Command{
 	Short: "Run the acl manager once, it will sync the follower with the primary",
 	Run: func(cmd *cobra.Command, args []string) {
 		ctx := cmd.Context()
-		aclManager := aclmanager.New(viper.GetString("address"), viper.GetString("username"), viper.GetString("password"), viper.GetBool("aclfile"))
-		function, err := aclManager.CurrentFunction(ctx)
+		mgr := aclmanager.New(viper.GetString("address"), viper.GetString("username"), viper.GetString("password"), viper.GetBool("aclfile"))
+		defer mgr.Close()
+		function, err := mgr.CurrentFunction(ctx)
 		if err != nil {
-			slog.Warn("unable to check if it's a Primary", "message", err)
+			slog.Error("Unable to check if node is primary", "error", err)
 			os.Exit(1)
 		}
 		if function == aclmanager.Follower {
-			primary, err := aclManager.Primary(ctx)
+			primary, err := mgr.Primary(ctx)
 			if err != nil {
-				slog.Warn("unable to find Primary", "message", err)
+				slog.Error("Unable to find Primary", "message", err)
 				os.Exit(1)
 			}
 			var added, deleted []string
-			added, deleted, err = aclManager.SyncAcls(ctx, primary)
+			added, deleted, err = mgr.SyncAcls(ctx, primary)
 			if err != nil {
-				slog.Warn("unable to sync acls from Primary", "message", err)
+				slog.Error("Unable to sync acls from Primary", "message", err)
 				os.Exit(1)
 			}
 			slog.Info("Synced acls from Primary", "added", added, "deleted", deleted)

configs/docker/vault/setup.sh → configs/development/vault/setup.sh

@@ -5,7 +5,7 @@ set -x
 setup_vault(){
     # Wait for Vault server to be up
     echo "Waiting for Vault to start..."
-    while ! nc -z localhost 8200; do
+    while ! nc -z vault 8200; do
       sleep 1
     done