Fixes #38: allow deletion/cancellation of jobs (#39)

* api: Shell2HttpAPI.delete method
* tests: TestDeletion testcase
* example: add deletion.py
* docs: info about deletion.py

Author: eshaan7

README.md

@@ -9,7 +9,7 @@ _For urgent issues and priority support, visit [https://xscode.com/eshaan7/flask
 [![codecov](https://codecov.io/gh/Eshaan7/Flask-Shell2HTTP/branch/master/graph/badge.svg?token=UQ43PYQPMR)](https://codecov.io/gh/Eshaan7/flask-shell2http/)
 [![CodeFactor](https://www.codefactor.io/repository/github/eshaan7/flask-shell2http/badge)](https://www.codefactor.io/repository/github/eshaan7/flask-shell2http)
 <a href="https://lgtm.com/projects/g/Eshaan7/Flask-Shell2HTTP/context:python">
-  <img alt="Language grade: Python" src="https://img.shields.io/lgtm/grade/python/g/Eshaan7/Flask-Shell2HTTP.svg?logo=lgtm&logoWidth=18"/>
+<img alt="Language grade: Python" src="https://img.shields.io/lgtm/grade/python/g/Eshaan7/Flask-Shell2HTTP.svg?logo=lgtm&logoWidth=18"/>
 </a>
 
 A minimalist [Flask](https://github.com/pallets/flask) extension that serves as a RESTful/HTTP wrapper for python's subprocess API.
@@ -18,23 +18,20 @@ A minimalist [Flask](https://github.com/pallets/flask) extension that serves as
 - Execute pre-defined shell commands asynchronously and securely via flask's endpoints with dynamic arguments, file upload, callback function capabilities.
 - Designed for binary to binary/HTTP communication, development, prototyping, remote control and [more](https://flask-shell2http.readthedocs.io/en/stable/Examples.html).
 
-
 ## Use Cases
 
 - Set a script that runs on a succesful POST request to an endpoint of your choice. See [Example code](examples/run_script.py).
 - Map a base command to an endpoint and pass dynamic arguments to it. See [Example code](examples/basic.py).
 - Can also process multiple uploaded files in one command. See [Example code](examples/multiple_files.py).
 - This is useful for internal docker-to-docker communications if you have different binaries distributed in micro-containers. See [real-life example](https://github.com/intelowlproject/IntelOwl/blob/master/integrations/static_analyzers/app.py).
-- You can define a callback function/ use signals to listen for process completion. See [Example code](examples/with_callback.py). 
-  * Maybe want to pass some additional context to the callback function ? 
-  * Maybe intercept on completion and update the result ? See [Example code](examples/custom_save_fn.py)
+- You can define a callback function/ use signals to listen for process completion. See [Example code](examples/with_callback.py).
+  - Maybe want to pass some additional context to the callback function ?
+  - Maybe intercept on completion and update the result ? See [Example code](examples/custom_save_fn.py)
 - You can also apply [View Decorators](https://flask.palletsprojects.com/en/1.1.x/patterns/viewdecorators/) to the exposed endpoint. See [Example code](examples/with_decorators.py).
-- Currently, all commands run asynchronously (default timeout is 3600 seconds), so result is not available directly. An option _may_ be provided for this in future releases for commands that return immediately.
 
 > Note: This extension is primarily meant for executing long-running
 > shell commands/scripts (like nmap, code-analysis' tools) in background from an HTTP request and getting the result at a later time.
 
-
 ## Documentation
 
 [![Documentation Status](https://readthedocs.org/projects/flask-shell2http/badge/?version=latest)](https://flask-shell2http.readthedocs.io/en/latest/?badge=latest)
@@ -43,14 +40,13 @@ A minimalist [Flask](https://github.com/pallets/flask) extension that serves as
 - I also highly recommend the [Examples](https://flask-shell2http.readthedocs.io/en/stable/Examples.html) section.
 - [CHANGELOG](https://github.com/eshaan7/Flask-Shell2HTTP/blob/master/.github/CHANGELOG.md).
 
-
 ## Quick Start
 
 ##### Dependencies
 
-* Python: `>=v3.6`
-* [Flask](https://pypi.org/project/Flask/)
-* [Flask-Executor](https://pypi.org/project/Flask-Executor)
+- Python: `>=v3.6`
+- [Flask](https://pypi.org/project/Flask/)
+- [Flask-Executor](https://pypi.org/project/Flask-Executor)
 
 ##### Installation
 
@@ -109,9 +105,9 @@ returns JSON,
 
 ```json
 {
-   "key": "ddbe0a94",
-   "result_url": "http://localhost:4000/commands/saythis?key=ddbe0a94&wait=false",
-   "status": "running"
+  "key": "ddbe0a94",
+  "result_url": "http://localhost:4000/commands/saythis?key=ddbe0a94&wait=false",
+  "status": "running"
 }
 ```
 
@@ -131,11 +127,10 @@ Returns result in JSON,
   "end_time": 1593019807.782958,
   "process_time": 0.00748753547668457,
   "returncode": 0,
-  "error": null,
+  "error": null
 }
 ```
 
-
 ## Inspiration
 
 This was initially made to integrate various command-line tools easily with [Intel Owl](https://github.com/intelowlproject/IntelOwl), which I am working on as part of Google Summer of Code.

docs/source/Examples.md

@@ -9,3 +9,4 @@ I have created some example python scripts to demonstrate various use-cases. The
 - [with_signals.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/with_signals.py): Using [Flask Signals](https://flask.palletsprojects.com/en/1.1.x/signals/) as callback function.
 - [with_decorators.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/with_decorators.py): Shows how to apply [View Decorators](https://flask.palletsprojects.com/en/1.1.x/patterns/viewdecorators/) to the exposed endpoint. Useful in case you wish to apply authentication, caching, etc. to the endpoint.
 - [custom_save_fn.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/custom_save_fn.py): There may be cases where the process doesn't print result to standard output but to a file/database. This example shows how to pass additional context to the callback function, intercept the future object after completion and update it's result attribute before it's ready to be consumed.
+- [deletion.py](https://github.com/Eshaan7/Flask-Shell2HTTP/blob/master/examples/deletion.py): Example demonstrating how to request cancellation/deletion of an already running job.

docs/source/index.rst

@@ -23,7 +23,6 @@ A minimalist Flask_ extension that serves as a RESTful/HTTP wrapper for python's
 - This is useful for internal docker-to-docker communications if you have different binaries distributed in micro-containers.
 - You can define a callback function/ use signals to listen for process completion.
 - You can also apply View Decorators to the exposed endpoint.
-- Currently, all commands run asynchronously (default timeout is 3600 seconds), so result is not available directly. An option _may_ be provided for this in future release.
 
    `Note: This extension is primarily meant for executing long-running 
    shell commands/scripts (like nmap, code-analysis' tools) in background from an HTTP request and getting the result at a later time.`

examples/deletion.py

@@ -0,0 +1,29 @@
+# web imports
+from flask import Flask
+from flask_executor import Executor
+from flask_shell2http import Shell2HTTP
+
+# Flask application instance
+app = Flask(__name__)
+
+# application factory
+executor = Executor(app)
+shell2http = Shell2HTTP(app, executor)
+
+
+shell2http.register_command(
+    endpoint="sleep",
+    command_name="sleep",
+)
+
+
+# Test Runner
+if __name__ == "__main__":
+    app.testing = True
+    c = app.test_client()
+    # request new process
+    r1 = c.post("/sleep", json={"args": ["10"], "force_unique_key": True})
+    print(r1)
+    # request cancellation
+    r2 = c.delete(f"/sleep?key={r1.get_json()['key']}")
+    print(r2)

flask_shell2http/api.py

@@ -28,7 +28,7 @@
 
 class Shell2HttpAPI(MethodView):
     """
-    ``Flask.MethodView`` that registers ``GET`` and ``POST``
+    ``Flask.MethodView`` that creates ``GET``, ``POST`` and ``DELETE``
     methods for a given endpoint.
     This is invoked on ``Shell2HTTP.register_command``.
 
@@ -37,6 +37,7 @@ class Shell2HttpAPI(MethodView):
 
     def get(self):
         """
+        Get report by job key.
         Args:
             key (str):
                 - Future key
@@ -138,6 +139,41 @@ def post(self):
                 response_dict["result_url"] = self.__build_result_url(key)
             return make_response(jsonify(response_dict), HTTPStatus.BAD_REQUEST)
 
+    def delete(self):
+        """
+        Cancel (if running) and delete job by job key.
+        Args:
+            key (str):
+                - Future key
+        """
+        try:
+            key = request.args.get("key")
+            logger.info(
+                f"Job: '{key}' --> deletion requested. "
+                f"Requester: '{request.remote_addr}'."
+            )
+            if not key:
+                raise Exception("No key provided in arguments.")
+
+            # get the future object
+            future: Future = self.executor.futures._futures.get(key)
+            if not future:
+                raise JobNotFoundException(f"No job exists for key: '{key}'.")
+
+            # cancel and delete from memory
+            future.cancel()
+            self.executor.futures.pop(key)
+
+            return make_response({}, HTTPStatus.NO_CONTENT)
+
+        except JobNotFoundException as e:
+            logger.error(e)
+            return make_response(jsonify(error=str(e)), HTTPStatus.NOT_FOUND)
+
+        except Exception as e:
+            logger.error(e)
+            return make_response(jsonify(error=str(e)), HTTPStatus.BAD_REQUEST)
+
     @classmethod
     def __build_result_url(cls, key: str) -> str:
         return f"{request.base_url}?key={key}&wait=false"

tests/test_deletion.py

@@ -0,0 +1,36 @@
+from examples.deletion import app
+
+from tests._utils import CustomTestCase
+
+
+class TestDeletion(CustomTestCase):
+    uri = "/sleep"
+
+    def create_app(self):
+        app.config["TESTING"] = True
+        return app
+
+    def test_delete__204(self):
+        # create command process
+        r1 = self.client.post(self.uri, json={"args": ["10"], "force_unique_key": True})
+        r1_json = r1.get_json()
+        self.assertStatus(r1, 202)
+        # request cancellation: correct key
+        r2 = self.client.delete(f"{self.uri}?key={r1_json['key']}")
+        self.assertStatus(r2, 204)
+
+    def test_delete__400(self):
+        # create command process
+        r1 = self.client.post(self.uri, json={"args": ["10"], "force_unique_key": True})
+        self.assertStatus(r1, 202)
+        # request cancellation: no key
+        r2 = self.client.delete(f"{self.uri}?key=")
+        self.assertStatus(r2, 400)
+
+    def test_delete__404(self):
+        # create command process
+        r1 = self.client.post(self.uri, json={"args": ["10"], "force_unique_key": True})
+        self.assertStatus(r1, 202)
+        # request cancellation: invalid key
+        r2 = self.client.delete(f"{self.uri}?key=abcdefg")
+        self.assertStatus(r2, 404)