Merge branch 'task_scheduler' into main

Author: danyi1212

.idea/runConfigurations/createtask.xml

@@ -30,6 +30,7 @@ Features
 - Authorize against **Active Directory** using `ldap3 <https://ldap3.readthedocs.io/en/latest/>`_ package
 - Manage **LDAP Connections** for easy integrations
 - Debug using `django-debug-toolbar <https://django-debug-toolbar.readthedocs.io/en/latest/>`_
+- **NEW** Create Task Schedulers for Django management commands
 
 Quick Start
 -----------

.idea/runConfigurations/runserver.xml

@@ -0,0 +1,64 @@
+
+Create Task Scheduler Jobs
+==========================
+
+It is usually necessary to **execute tasks** from you project on a schedule.
+This module has a shortcut to create scheduled jobs for **Django management commands** in the **Windows Task Scheduler**.
+
+.. warning::
+    This feature requires the installation of the ``pywin32`` module.
+    Install it with ``pip install pywin32``
+
+Create a task
+-------------
+
+Creating a new task is done with the ``createtask`` command.
+
+For example, lets say you want to run the following command every hour::
+
+$ py manage.py say_hello --new-users
+
+You can create a schedule with this command::
+
+$ py manage.py createtask "say_hello --new-users" -i hours=1
+
+Now the following command will be executed every hour by the Windows Task Scheduler.
+
+Using predefined tasks
+----------------------
+
+Included with this module are some **predefined tasks** for some Django and Third-Party app management commands.
+Those commands can be created using the ``--predefined`` or ``-p`` argument
+
+:clearsessions:
+    Clear expired sessions from database, once a week::
+
+    $ py manage.py createtask clearsessions -p
+
+.. seealso::
+    See more at https://docs.djangoproject.com/en/3.1/ref/django-admin/#django-admin-clearsessions
+
+:clean_duplicate_history:
+    Clean duplicate history records from all models with history every 3 hours (from django-simple-history)::
+
+    $ py manage.py createtask clean_duplicate_history -p
+
+:clean_old_history:
+    Clean history records older then 30 days from all models with history every day (from django-simple-history)::
+
+    $ py manage.py createtask clean_old_history -p
+
+
+.. seealso::
+    See more at https://django-simple-history.readthedocs.io/en/latest/utils.html#utils
+
+:process_tasks:
+    Worker for background tasks processing (from django-background-tasks)::
+
+    $ py manage.py createtask process_tasks -p
+
+
+You can also create **multiple workers** by specifying different names with ``--name`` argument.
+
+.. seealso::
+    See more at https://django-background-tasks.readthedocs.io/en/latest/#running-tasks

.idea/runConfigurations/test.xml

@@ -15,6 +15,7 @@ Features
 - Authorize against Active Directory using `ldap3 <https://ldap3.readthedocs.io/en/latest/>`_ package
 - Manage LDAP connections for easy integrations
 - Debug using `django-debug-toolbar <https://django-debug-toolbar.readthedocs.io/en/latest/>`_
+- **NEW** Create Task Schedulers for Django management commands
 
 .. toctree::
    :maxdepth: 1
@@ -30,6 +31,7 @@ Features
    :caption: How-to Guides
 
    howto/serve_static
+   howto/create_tasks
    howto/custom_user_fields
    howto/using_ldap_manager
    howto/manage_secrets

README.rst

@@ -8,6 +8,7 @@ Change Log
 Release date: coming soon
 
 - **ADDED**: LDAPUserManager for manually creating users from LDAP.
+- **ADDED**: ``createtask`` management command for creating Task Scheduler jobs.
 - **IMPROVED**: LDAP Settings for Group Membership check propagate to one another.
 - **MODIFIED**: Increased the default ``WAUTH_RESYNC_DELTA`` to every 1 day.
 

docs/source/howto/create_tasks.rst

@@ -21,4 +21,27 @@ Arguments
 
 .. warning::
     In order for the ``web.config`` files to work correctly, you will need to **unlock** some IIS Configuration Section.
-    See the **Install and Setup IIS** section at :doc:`../installation/installation` docs.
+    See the **Install and Setup IIS** section at :doc:`../installation/installation` docs.
+
+createtask
+----------
+
+Add a management command to Windows Task Scheduler.
+
+Arguments
+    * **command** Management command, wrapped with "command".
+    * **--predefined**, **-p** Create from a predefined task.
+    * **--name**, **-n** Task name.
+    * **--desc**, **-d** Task description.
+    * **--identity**, **-u** Task principal identity (default: "NT Authority\\LocalSystem").
+    * **--folder**, **-f** Task folder location (default: Project's name).
+    * **--interval**, **-i** Task execution interval as timedelta kwargs, e.g. "days=1,hours=12.5".
+    * **--random**, **-r** Randomize execution time as timedelta kwargs, e.g. "days=1,hours=12.5".
+    * **--timeout**, **-t** Execution time limit as timedelta kwargs, e.g. "days=1,hours=12.5" (default: 1 hour).
+    * **--priority** Task priority https://docs.microsoft.com/en-us/windows/win32/taskschd/tasksettings-priority
+
+Predefined tasks
+    * **clearsessions** Clear sessions from database every week.
+    * **clean_duplicate_history** Clean duplicate history records from all models with history every 3 hours (from django-simple-history).
+    * **clean_old_history** Clean history records older then 30 days from all models with history every day (from django-simple-history).
+    * **process_tasks** Worker for background tasks processing (from django-background-tasks).

docs/source/index.rst

@@ -0,0 +1,85 @@
+import os
+from argparse import ArgumentTypeError
+
+from django.conf import settings
+from django.core.management import BaseCommand, CommandParser, CommandError
+from django.utils import timezone
+from pythoncom import com_error
+
+from windows_auth.scheduler import create_task_definition, LOCAL_SERVICE, add_schedule_trigger, register_task
+from windows_auth.predefined_tasks import clear_sessions_task, clean_duplicate_history_task, clean_old_history_task, process_tasks_task
+
+
+def parse_datetime(string):
+    result = {}
+    for arg in string.split(","):
+        if arg:
+            try:
+                # split key & value
+                key, value = arg.split("=", 2)
+            except ValueError:
+                raise ArgumentTypeError("Argument must be in \"key=value\" format.")
+
+            try:
+                # parse to float
+                result[key] = float(value)
+            except ValueError as e:
+                raise ArgumentTypeError(e)
+    try:
+        # create time delta
+        return timezone.timedelta(**result)
+    except TypeError as e:
+        raise ArgumentTypeError(str(e).replace("__new__()", "timedelta()"))
+
+
+PREDEFINED_TASKS = {
+    "clearsessions": clear_sessions_task,
+    "clean_duplicate_history": clean_duplicate_history_task,
+    "clean_old_history": clean_old_history_task,
+    "process_tasks": process_tasks_task,
+}
+
+
+class Command(BaseCommand):
+    help = "Add a management command to Windows Task Scheduler."
+
+    def add_arguments(self, parser: CommandParser):
+        parser.add_argument("command", help="Management command, wrapped with \"command\"")
+        parser.add_argument("-p", "--predefined", action="store_true",
+                            help=f"Create from a predefined task {tuple(PREDEFINED_TASKS.keys())}")
+        parser.add_argument("-n", "--name", type=str, help="Task name")
+        parser.add_argument("-d", "--desc", type=str, default="", help="Task description")
+        parser.add_argument("-u", "--identity", type=str, default=LOCAL_SERVICE, help="Task principle identity"),
+        parser.add_argument("-f", "--folder", type=str, default=os.path.basename(settings.BASE_DIR),
+                            help="Task folder location")
+        parser.add_argument("-i", "--interval", type=parse_datetime,
+                            help="Task interval as timedelta kwargs, e.g. \"days=1,hours=12.5\".")
+        parser.add_argument("-r", "--random", type=parse_datetime,
+                            help="Randomize execution time as timedelta kwargs, e.g. \"days=1,hours=12.5\".")
+        parser.add_argument("-t", "--timeout", type=parse_datetime, default="hours=1",
+                            help="Execution Time Limit as timedelta kwargs, e.g. \"days=1,hours=12.5\".")
+        parser.add_argument("--priority", type=int, default=3,
+                            help="Task Priority "
+                                 "https://docs.microsoft.com/en-us/windows/win32/taskschd/tasksettings-priority")
+
+    def handle(self, command="", predefined=False, name=None, desc="", identity=LOCAL_SERVICE, folder=None,
+               interval=None, random=None, timeout=None, priority=None, **options):
+        try:
+            if predefined:
+                # predefined tasks
+                if command in PREDEFINED_TASKS:
+                    create_task = PREDEFINED_TASKS[command]
+                    create_task(command=command, name=name, desc=desc, identity=identity, folder=folder,
+                                interval=interval, random=random, timeout=timeout, priority=priority)
+                else:
+                    raise CommandError(f"Predefined task for \"{command}\" does not exist.")
+            else:
+                # create task definition
+                task_def = create_task_definition(command, description=desc, priority=priority, timeout=timeout)
+                # add trigger
+                if interval:
+                    add_schedule_trigger(task_def, interval, random=random)
+                # register task
+                register_task(task_def, name or command.split(" ", 1)[0], folder=folder, username=identity)
+        except com_error as e:
+            raise CommandError("Failed to register task. Did you run as administrator?\n" + str(e))

docs/source/reference/change_log.rst

@@ -0,0 +1,69 @@
+from django.utils import timezone
+
+from windows_auth.scheduler import create_task_definition, add_schedule_trigger, register_task
+
+
+def _get_options(options: dict, *args) -> dict:
+    """
+    Filter specific keys from a dictionary
+    :param options: Source dictionary
+    :param args: Keys to keep
+    :return: Filtered dictionary
+    """
+    return {
+        key: value
+        for key, value in options.items()
+        if key in args
+    }
+
+
+def clear_sessions_task(**options):
+    """
+    Clear sessions from database every week
+    """
+    task_def = create_task_definition("clearsessions",
+                                      description=options.get("desc") or "Clear expired sessions from database",
+                                      **_get_options(options, "priority", "timeout"))
+    add_schedule_trigger(task_def, timezone.timedelta(weeks=1), random=timezone.timedelta(1))
+    register_task(task_def, options.get("name") or "Clear Sessions",
+                  **_get_options(options, "folder", "username", "password"))
+
+
+def clean_duplicate_history_task(**options):
+    """
+    Clean duplicate history records from all models with history every 3 hours (from django-simple-history).
+    """
+    interval = options.get("interval") or timezone.timedelta(hours=3)
+    task_def = create_task_definition(f"clean_duplicate_history -m {interval.seconds / 60} --auto",
+                                      description=options.get(
+                                          "desc") or "Clean duplicate history records from database",
+                                      **_get_options(options, "priority", "timeout"))
+    add_schedule_trigger(task_def, interval)
+    register_task(task_def, options.get("name") or "Clean Duplicate History",
+                  **_get_options(options, "folder", "username", "password"))
+
+
+def clean_old_history_task(**options):
+    """
+    Clean history records older then 30 days from all models with history every day (from django-simple-history).
+    """
+    task_def = create_task_definition("clean_old_history --auto",
+                                      description=options.get("desc") or "Clean old history records from database",
+                                      **_get_options(options, "priority", "timeout"))
+    add_schedule_trigger(task_def, timezone.timedelta(days=1))
+    register_task(task_def, options.get("name") or "Clean Old History",
+                  **_get_options(options, "folder", "username", "password"))
+
+
+def process_tasks_task(**options):
+    """
+    Worker for background tasks processing (from django-background-tasks)
+    """
+    interval = options.get("interval") or timezone.timedelta(hours=1)
+    task_def = create_task_definition(f"process_tasks --log-std --duration {interval.seconds}",
+                                      description=options.get("desc") or "Background tasks worker",
+                                      **_get_options(options, "priority", "timeout"))
+    add_schedule_trigger(task_def, interval)
+    task = register_task(task_def, options.get("name") or "Process background tasks",
+                         **_get_options(options, "folder", "username", "password"))
+    task.Run(None)  # start immediately