overlord: implement user group system

In order to better support groups. We ditched the htpasswd format and
user a sqlite database instead.

Author: aitjcize

README.rst

@@ -33,12 +33,50 @@ Run ``make`` in the project root directory.  ``make`` builds the overlord daemon
 
 Basic Deployment
 ----------------
-1. On a server with public IP (to be used as a proxy), run ``overlordd -port 9000`` to start the server (if ``-port`` is not specified, default port is 80 or 443 depends on whether TLS is enabled or not).
-2. On a client machine, run ``ghost SERVER_IP`` or ``ghost.py SERVER_IP``.  ``ghost`` and ``ghost.py`` are functional equivalent except one is written in Python, and the other is written in Go.
-3. Browse http://SERVER_IP:9000 and you will see the overlord web dashboard.  The default user/password is ``overlord/cros``.  To change the password, please follow the `Overlord advanced deployment guide <https://github.com/aitjcize/Overlord/blob/master/docs/deployment.rst#changing-default-password>`_.
+1. On a server with public IP (to be used as a proxy), initialize the database by running ``overlordd -init -db-path overlord.db``. You'll be prompted to create an admin username and password.
+2. Start the server with ``overlordd -port 9000 -db-path overlord.db`` (if ``-port`` is not specified, default port is 80 or 443 depends on whether TLS is enabled or not).
+3. On a client machine, run ``ghost SERVER_IP`` or ``ghost.py SERVER_IP``.  ``ghost`` and ``ghost.py`` are functional equivalent except one is written in Python, and the other is written in Go.
+4. Browse http://SERVER_IP:9000 and you will see the overlord web dashboard. Log in with the admin credentials you created during initialization.
 
-For testing purpose, you can run both server and client on the same machine, then browse http://localhost:9000 instead.  If you want to disable authentication of the web dashboard, you could add the ``-noauth`` option when starting ``overlordd``.
-Overlord server supports a lot of features such as TLS encryption, client auto upgrade.  For setting up these, please refer to the `Overlord advanced deployment guide <https://github.com/aitjcize/Overlord/blob/master/docs/deployment.rst>`_.
+For testing purpose, you can run both server and client on the same machine, then browse http://localhost:9000 instead.  Overlord server supports a lot of features such as TLS encryption, client auto upgrade.  For setting up these, please refer to the `Overlord advanced deployment guide <https://github.com/aitjcize/Overlord/blob/master/docs/deployment.rst>`_.
+
+User and Group Management
+------------------------
+Overlord uses a SQLite database for user and group management. The database is stored in the file specified by the ``-db-path`` parameter (default: ``overlord.db``).
+
+Before starting the server for the first time, you must initialize the database with the ``-init`` flag:
+
+.. code-block:: bash
+
+   overlordd -init -db-path overlord.db
+
+This command will:
+1. Create the database schema
+2. Generate a secure JWT secret for authentication
+3. Prompt you to create an admin user with a custom username and password
+4. Create the admin group
+
+You can manage users and groups through the web interface by navigating to:
+- Users: ``/users``
+- Groups: ``/groups``
+
+Only administrators can create, modify, and delete users and groups.
+
+Allowlist Format
+---------------
+Overlord supports access control through allowlists that specify which users can access which clients. The allowlist format has been enhanced to support both users and groups with the following prefixes:
+
+- ``u/username`` - Grants access to a specific user
+- ``g/groupname`` - Grants access to all members of a specific group
+- ``anyone`` - Grants access to all authenticated users
+
+When running a Ghost client, you can specify the allowlist using the ``--allowlist`` parameter:
+
+.. code-block:: bash
+
+   ghost --allowlist u/user1,u/user2,g/group1 SERVER_IP
+
+If no prefix is provided for an entity, it's assumed to be a username and will be automatically prefixed with ``u/``.
 
 Usage
 -----
@@ -114,7 +152,52 @@ Besides from the web interface, a command line interface is also provided.  The
    
    2016-03-08 17:57:00 (37.5 MB/s) - ‘index.html’ saved [419/419]
 
+7. User and group management with admin subcommand:
+
+.. code-block:: bash
 
+   # List all users
+   % ovl admin list-users
+   USERNAME             ADMIN      GROUPS
+   --------------------------------------------------
+   admin                Yes        admin
+   user1                No         testers
+
+   # Add a new user
+   % ovl admin add-user username password
+   User 'username' created successfully
+
+   # Add user to admin group
+   % ovl admin add-user-to-group username admin
+   User 'username' added to group 'admin' successfully
+
+   # List groups
+   % ovl admin list-groups
+   GROUP NAME           USER COUNT
+   ------------------------------
+   admin                2
+   testers              1
+
+   # View users in a group
+   % ovl admin list-group-users admin
+   Users in group 'admin':
+     - admin
+     - username
+
+REST API
+--------
+Overlord provides a REST API for managing users and groups:
+
+- GET /api/users - List all users
+- POST /api/users - Create a new user
+- DELETE /api/users/{username} - Delete a user
+- PUT /api/users/{username}/password - Update a user's password
+- GET /api/groups - List all groups
+- POST /api/groups - Create a new group
+- DELETE /api/groups/{groupname} - Delete a group
+- POST /api/groups/{groupname}/users - Add a user to a group
+- DELETE /api/groups/{groupname}/users/{username} - Remove a user from a group
+- GET /api/groups/{groupname}/users - List all users in a group
 
 Disclaimer
 ----------

bin/jwt-secret

@@ -5,49 +5,147 @@
 package main
 
 import (
+	"bufio"
 	"flag"
 	"fmt"
 	"os"
+	"path/filepath"
+	"strings"
+	"syscall"
 
 	"github.com/aitjcize/Overlord/overlord"
+	"golang.org/x/term"
 )
 
-var bindAddr = flag.String("bind", "0.0.0.0", "specify alternate bind address")
-var port = flag.Int("port", 0,
-	"alternate port listen instead of standard ports (http:80, https:443)")
-var lanDiscInterface = flag.String("lan-disc-iface", "",
-	"the network interface used for broadcasting LAN discovery packets")
-var noLanDisc = flag.Bool("no-lan-disc", false,
-	"disable LAN discovery broadcasting")
-var tlsCerts = flag.String("tls", "",
-	"TLS certificates in the form of 'cert.pem,key.pem'. Empty to disable.")
-var noLinkTLS = flag.Bool("no-link-tls", false,
-	"disable TLS between ghost and overlord. Only valid when TLS is enabled.")
-var htpasswdPath = flag.String("htpasswd-path", "overlord.htpasswd",
-	"the path to the .htpasswd file. Required for authentication.")
-var jwtSecretPath = flag.String("jwt-secret-path", "jwt-secret",
-	"Path to the file containing the JWT secret. Required for authentication.")
+var (
+	bindAddr = flag.String("bind",
+		"0.0.0.0", "specify alternate bind address")
+	port = flag.Int("port",
+		0, "alternate port listen instead of standard ports (http:80, https:443)")
+	lanDiscInterface = flag.String("lan-disc-iface",
+		"", "the network interface used for broadcasting LAN discovery packets")
+	noLanDisc = flag.Bool("no-lan-disc",
+		false, "disable LAN discovery broadcasting")
+	tlsCerts = flag.String("tls",
+		"", "TLS certificates in the form of 'cert.pem,key.pem'. Empty to disable.")
+	noLinkTLS = flag.Bool("no-link-tls",
+		false, "disable TLS between ghost and overlord. Only valid when TLS is enabled.")
+	dbPath = flag.String("db-path",
+		"overlord.db", "the path to the SQLite database file for user, group, and authentication data")
+	initializeDB = flag.Bool("init",
+		false, "Initialize the database with a custom admin user and password and generate a JWT secret")
+	adminUser = flag.String("admin-user",
+		"", "Admin username for database initialization (only used with -init)")
+	adminPass = flag.String("admin-pass",
+		"", "Admin password for database initialization (only used with -init)")
+)
 
 func usage() {
-	fmt.Fprintf(os.Stderr, "Usage: overlordd [OPTIONS]\n")
+	prog := filepath.Base(os.Args[0])
+	fmt.Fprintf(os.Stderr, "Usage: %s [options]\n", prog)
+	fmt.Fprintln(os.Stderr, "Options:")
 	flag.PrintDefaults()
-	os.Exit(2)
+	os.Exit(1)
+}
+
+func promptForInput(prompt string) string {
+	reader := bufio.NewReader(os.Stdin)
+	fmt.Print(prompt)
+	input, _ := reader.ReadString('\n')
+	return strings.TrimSpace(input)
+}
+
+func promptForPassword(prompt string) string {
+	fmt.Print(prompt)
+	password, err := term.ReadPassword(int(syscall.Stdin))
+	fmt.Println() // Add a newline after the password input
+
+	if err != nil {
+		panic(err)
+	}
+	return string(password)
+}
+
+func initializeDatabase(dbPath string) error {
+	adminUsername := *adminUser
+	adminPassword := *adminPass
+
+	// If admin username is not provided via command line, prompt for it
+	if adminUsername == "" {
+		adminUsername = promptForInput("Enter admin username [admin]: ")
+		if adminUsername == "" {
+			adminUsername = "admin"
+		}
+	}
+
+	// If admin password is not provided via command line, prompt for it
+	if adminPassword == "" {
+		for {
+			adminPassword = promptForPassword("Enter admin password: ")
+			if adminPassword == "" {
+				fmt.Println("Password cannot be empty, please try again.")
+				continue
+			}
+			break
+		}
+	} else if adminPassword == "" {
+		return fmt.Errorf("password cannot be empty")
+	}
+
+	dbManager := overlord.NewDatabaseManager(dbPath)
+
+	err := dbManager.Initialize(adminUsername, adminPassword)
+	if err != nil {
+		return fmt.Errorf("failed to initialize database: %v", err)
+	}
+
+	fmt.Println("Database initialization complete.")
+	fmt.Println("You can now start the server without the -init flag.")
+	return nil
+}
+
+func checkDatabaseInitialized(dbPath string) bool {
+	// Simple check - if the file exists and has data, consider it initialized
+	info, err := os.Stat(dbPath)
+	if err != nil || info.Size() == 0 {
+		return false
+	}
+	return true
 }
 
 func main() {
-	flag.Usage = usage
 	flag.Parse()
 
-	// Validate required flags
-	if *htpasswdPath == "" {
-		fmt.Fprintf(os.Stderr, "Error: -htpasswd-path is required\n")
+	if len(flag.Args()) > 0 {
+		fmt.Fprintf(os.Stderr, "Error: unknown argument: %s\n", flag.Args()[0])
 		usage()
 	}
-	if *jwtSecretPath == "" {
-		fmt.Fprintf(os.Stderr, "Error: -jwt-secret-path is required\n")
+
+	// Validate required flags
+	if *dbPath == "" {
+		fmt.Fprintf(os.Stderr, "Error: -db-path is required\n")
 		usage()
 	}
 
+	// Initialize the database if requested
+	if *initializeDB {
+		if checkDatabaseInitialized(*dbPath) {
+			fmt.Fprintf(os.Stderr, "Error: Database already initialized\n")
+			os.Exit(1)
+		}
+		if err := initializeDatabase(*dbPath); err != nil {
+			fmt.Fprintf(os.Stderr, "Error initializing database: %v\n", err)
+			os.Exit(1)
+		}
+		os.Exit(0)
+	}
+
+	// Check if the database is initialized
+	if !checkDatabaseInitialized(*dbPath) {
+		fmt.Fprintf(os.Stderr, "Error: Database not initialized. Run with -init to initialize\n")
+		os.Exit(1)
+	}
+
 	overlord.StartOverlord(*bindAddr, *port, *lanDiscInterface, !*noLanDisc,
-		*tlsCerts, !*noLinkTLS, *htpasswdPath, *jwtSecretPath)
+		*tlsCerts, !*noLinkTLS, *dbPath)
 }