CLI: create function: support single-file functions (#474)

* CLI: create function: single-file functions

Make `--handler` unnecessary when creating single-file functions.
The base-image will try to guess the handler at function creation.
This is language dependent.

Function creation will fail for base-images not supporting this feature.

Also, update docs and base-image versions

* Bump base-image versions

* Error out when source is a dir and no handler is set

Author: imikushin

docs/_guides/quickstart.md

@@ -90,13 +90,13 @@ $ dispatch create --file seed.yaml --work-dir examples/
 $ dispatch get images
    NAME   |                    URL                |  BASEIMAGE   |   STATUS    |         CREATED DATE
 ------------------------------------------------------------------------------------------------------------------------
-  nodejs6 | dispatchframework/nodejs-base:0.0.3   | nodejs6-base | READY       | Wed Dec  6 14:28:30 PST 2017
-  python3 | dispatchframework/python3-base:0.0.3  | python3-base | INITIALIZED | Wed Dec  6 14:28:30 PST 2017
+  nodejs  | dispatchframework/nodejs-base:0.0.7   | nodejs-base  | READY       | Wed Dec  6 14:28:30 PST 2017
+  python3 | dispatchframework/python3-base:0.0.7  | python3-base | INITIALIZED | Wed Dec  6 14:28:30 PST 2017
 
 $ dispatch get functions
     NAME   |  IMAGE  | STATUS |         CREATED DATE
 ------------------------------------------------------------
-  hello-js | nodejs6 | READY  | Wed Dec  6 14:29:05 PST 2017
+  hello-js | nodejs | READY  | Wed Dec  6 14:29:05 PST 2017
   hello-py | python3 | READY  | Wed Dec  6 14:28:52 PST 2017
 ```
 

docs/_guides/setup-service-acccount-authentication.md

@@ -49,10 +49,10 @@ use this private key to sign the generated JWT token.
 
 ```bash
 $ dispatch create -f seed.yaml --service-account example-svc-account --jwt-private-key ../example-user.key
-Created BaseImage: nodejs6-base
+Created BaseImage: nodejs-base
 Created BaseImage: python3-base
 Created BaseImage: powershell-base
-Created Image: nodejs6
+Created Image: nodejs
 Created Image: python3
 Created Image: powershell
 Created Function: hello-py
@@ -64,9 +64,9 @@ Created Secret: open-sesame
 $ dispatch get base-image --service-account example-svc-account --jwt-private-key ../example-user.key
        NAME       |                   URL                   | STATUS |         CREATED DATE
 ------------------------------------------------------------------------------------------------
-  python3-base    | vmware/dispatch-python3-base:0.0.2-dev1 | READY  | Sat Jan  1 14:40:18 PST 0000
-  nodejs6-base    | vmware/dispatch-nodejs6-base:0.0.2-dev1 | READY  | Sat Jan  1 14:40:18 PST 0000
-  powershell-base | vmware/dispatch-powershell-base:0.0.3   | READY  | Sat Jan  1 14:40:18 PST 0000
+  python3-base    | dispatchframework/python3-base:0.0.7    | READY  | Sat Jan  1 14:40:18 PST 0000
+  nodejs-base     | dispatchframework/nodejs-base:0.0.6     | READY  | Sat Jan  1 14:40:18 PST 0000
+  powershell-base | dispatchframework/powershell-base:0.0.8 | READY  | Sat Jan  1 14:40:18 PST 0000
 ```
 
 If you are directly calling the API instead of using the CLI, you need to create a JWT payload as follows:

docs/_specs/language-support/language-support.md

@@ -48,10 +48,8 @@ This directory contains the Dockerfile and any supporting files (but typically,
 
 The Dockerfile for the function image accepts build args (using `ARG` instruction):
 
-- **`IMAGE`** — the *image* the be used as the base for the function image:
-    ARG IMAGE
-    FROM ${IMAGE}
-- **`FUNCTION_SRC`**, defaults to `function.txt` — the file with the function source code, put into the function image build container AS IS by Dispatch function builder. 
+- **`IMAGE`** — the *image* the be used as the base for the *function image*
+- **`HANDLER`** — language specific handler function name (module path, fully qualified name, etc.) 
 
 Function builder expects the Dockerfile to have all the necessary instructions to build and install the function, so that when a container is run from the function image, the *Function Runtime API* (see below) is exposed on the port specified by `PORT` env var, which is 8080 by default. 
 
@@ -60,7 +58,7 @@ Function builder expects the Dockerfile to have all the necessary instructions t
 
 It’s a request/reply RPC-style function invocation API implemented with a simple HTTP server written in the language provided by the *base image*. All function invocations go through this API. How the invocations reach this API is FaaS specific and outside of the function runtime responsibilities. 
 
-When a container is normally run from a *function* image (without specifying commands), the main (and usually, the only) process in that container listens on the specified `PORT` and serves this API via HTTP. 
+When a container is normally run from a *function* image (without specifying commands), the main (and usually, the only) process in that container listens on the specified `PORT` and serves this API via HTTP, using the specified `HANDLER` function to handle incoming invocations. 
 
 It is the API server process’s responsibility to adequately react to OS signals (such as `SIGTERM`) and perform graceful shutdown, i.e. to stop accepting new requests and finish processing already accepted invocation(s) before exiting. 
 
@@ -123,18 +121,18 @@ Payloads must be JSON encodable.
 
 A **user** registers an existing docker image as a *base image* in Dispatch:
 
-    $ dispatch create base-image js dispatchframework/nodejs-base:0.0.2
+    $ dispatch create base-image js-base dispatchframework/nodejs-base:0.0.7
 
-Here, `dispatchframework/nodejs-base:0.0.2` is registered as the *base image* named `js`.
+Here, `dispatchframework/nodejs-base:0.0.7` is registered as the *base image* named `js-base`.
 
 
 ## Create an *Image*
 
 A **user** creates an *image* to be used for their functions, including any needed system packages and library dependencies:
 
-    $ dispatch create image js-deps js --runtime-deps ./package.json
+    $ dispatch create image js-deps js-base --runtime-deps ./package.json
 
-Here, the *image* named `js-deps` is created from the *base image* `js` adding dependencies from the manifest file `./package.json`. 
+Here, the *image* named `js-deps` is created from the *base image* `js-base` adding dependencies from the manifest file `./package.json`. 
 
 In order to create the *image*, **Dispatch image manager** does the following:
 
@@ -151,13 +149,13 @@ A **user** creates a *function* from a source file using the specified *image*:
 
     $ dispatch create function --image=js-deps hello1 . --handler=./hello.js
 
-Here, the *function* named `hello1` is created from the source file `./hello.js` using the image `js-deps` containing library packages that can be used by the *function*. 
+Here, the specified directory (`.`) is used as the root for function source files, *function* named `hello1` is created from `./hello.js`, using image `js-deps` containing library packages. 
 
 In order to create the function, **Dispatch function manager** does the following:
 
 1. creates (doesn’t run) a temporary container from the specified *image*
 2. copies the directory specified by the metadata label `io.dispatchframework.functionTemplate` (in our case, `/function-template`) from the container into a temporary directory
-3. copies the function source file into the same directory
+3. copies the function sources into the same directory
 4. builds the docker image from the temporary directory, using the *image* docker image as the value of `IMAGE` build argument
 5. registers the docker image as the *function* image.
 
@@ -176,10 +174,10 @@ In order to create the function, **Dispatch function manager** does the followin
 
 There is a selection of base-images implementing this spec in [dispatchframework](https://github.com/dispatchframework) organization on GitHub:
 
-- https://github.com/dispatchframework/nodejs-base-image
-- https://github.com/dispatchframework/python3-base-image
-- https://github.com/dispatchframework/powershell-base-image
-- https://github.com/dispatchframework/java-base-image
-- https://github.com/dispatchframework/clojure-base-image
+- [nodejs-base-image](https://github.com/dispatchframework/nodejs-base-image)
+- [python3-base-image](https://github.com/dispatchframework/python3-base-image)
+- [powershell-base-image](https://github.com/dispatchframework/powershell-base-image)
+- [java-base-image](https://github.com/dispatchframework/java-base-image)
+- [clojure-base-image](https://github.com/dispatchframework/clojure-base-image)
 
 Any one of these images can be used as an example of how to add a new language support to Dispatch.

e2e/tests/api_update.yaml

@@ -2,7 +2,7 @@ kind: API
 name: api-test-update
 authentication: public
 enabled: true
-function: func-nodejs6
+function: func-nodejs
 methods:
   - GET
 protocols:

e2e/tests/apis.bats

@@ -10,31 +10,31 @@ load variables
 
 @test "Create Images for test" {
 
-    run dispatch create base-image base-nodejs6 $DOCKER_REGISTRY/$BASE_IMAGE_NODEJS6 --language nodejs6
+    run dispatch create base-image base-nodejs $DOCKER_REGISTRY/$BASE_IMAGE_NODEJS6 --language nodejs
     assert_success
-    run_with_retry "dispatch get base-image base-nodejs6 --json | jq -r .status" "READY" 4 5
+    run_with_retry "dispatch get base-image base-nodejs --json | jq -r .status" "READY" 4 5
 
-    run dispatch create image nodejs6 base-nodejs6
+    run dispatch create image nodejs base-nodejs
     assert_success
-    run_with_retry "dispatch get image nodejs6 --json | jq -r .status" "READY" 8 5
+    run_with_retry "dispatch get image nodejs --json | jq -r .status" "READY" 8 5
 }
 
 @test "Create Functions for test" {
-    run dispatch create function --image=nodejs6 func-nodejs6 ${DISPATCH_ROOT}/examples/nodejs6 --handler=./hello.js
+    run dispatch create function --image=nodejs func-nodejs ${DISPATCH_ROOT}/examples/nodejs --handler=./hello.js
     echo_to_log
     assert_success
 
-    run_with_retry "dispatch get function func-nodejs6 --json | jq -r .status" "READY" 10 5
+    run_with_retry "dispatch get function func-nodejs --json | jq -r .status" "READY" 10 5
 
-    run dispatch create function --image=nodejs6 node-echo-back ${DISPATCH_ROOT}/examples/nodejs6 --handler=./debug.js
+    run dispatch create function --image=nodejs node-echo-back ${DISPATCH_ROOT}/examples/nodejs --handler=./debug.js
     echo_to_log
     assert_success
 
     run_with_retry "dispatch get function node-echo-back --json | jq -r .status" "READY" 10 5
 }
 
 @test "Test APIs with HTTP(S)" {
-    run dispatch create api api-test-http func-nodejs6 -m POST -p /http --auth public
+    run dispatch create api api-test-http func-nodejs -m POST -p /http --auth public
     echo_to_log
     assert_success
 
@@ -54,7 +54,7 @@ load variables
 }
 
 @test "Test APIs with HTTPS ONLY" {
-    run dispatch create api api-test-https-only func-nodejs6 -m POST --https-only -p /https-only --auth public
+    run dispatch create api api-test-https-only func-nodejs -m POST --https-only -p /https-only --auth public
     echo_to_log
     assert_success
     run_with_retry "dispatch get api api-test-https-only --json | jq -r .status" "READY" 6 5
@@ -72,7 +72,7 @@ load variables
 
 @test "Test APIs with Kong Plugins" {
 
-    run dispatch create api api-test func-nodejs6 -m GET -m DELETE -m POST -m PUT -p /hello --auth public
+    run dispatch create api api-test func-nodejs -m GET -m DELETE -m POST -m PUT -p /hello --auth public
     echo_to_log
     assert_success
     run_with_retry "dispatch get api api-test --json | jq -r .status" "READY" 6 5
@@ -120,7 +120,7 @@ load variables
 }
 
 @test "Test APIs with CORS" {
-    run dispatch create api api-test-cors func-nodejs6 -m POST -m PUT -p /cors --auth public --cors
+    run dispatch create api api-test-cors func-nodejs -m POST -m PUT -p /cors --auth public --cors
     echo_to_log
     assert_success
     run_with_retry "dispatch get api api-test-cors --json | jq -r .status" "READY" 10 5
@@ -133,7 +133,7 @@ load variables
 }
 
 @test "Test API Updates" {
-    run dispatch create api api-test-update func-nodejs6 -m GET -p /hello --auth public
+    run dispatch create api api-test-update func-nodejs -m GET -p /hello --auth public
     assert_success
     run_with_retry "dispatch get api api-test-update --json | jq -r .status" "READY" 6 5
 

e2e/tests/application.bats

@@ -25,21 +25,21 @@ load variables
     run dispatch get base-image
     assert_success
 
-    # Create base image "base-nodejs6"
-    run dispatch create base-image base-nodejs6 $DOCKER_REGISTRY/$BASE_IMAGE_NODEJS6 --language nodejs6
+    # Create base image "base-nodejs"
+    run dispatch create base-image base-nodejs $DOCKER_REGISTRY/$BASE_IMAGE_NODEJS6 --language nodejs
     assert_success
 
-    run_with_retry "dispatch get base-image base-nodejs6 --json | jq -r .status" "READY" 4 5
+    run_with_retry "dispatch get base-image base-nodejs --json | jq -r .status" "READY" 4 5
 }
 
 @test "Image creation" {
 
     run_with_retry "dispatch get image --json | jq -r length" 0 1 1
     assert_success
 
-    run dispatch create image foo-app-image base-nodejs6 --application foo-app
+    run dispatch create image foo-app-image base-nodejs --application foo-app
     assert_success
-    run dispatch create image bar-app-image base-nodejs6 --application bar-app
+    run dispatch create image bar-app-image base-nodejs --application bar-app
     assert_success
 
     # list image of foo app, only one returns
@@ -60,11 +60,11 @@ load variables
 
 @test "Secret creation" {
 
-    run dispatch create secret open-sesame -a foo-app ${DISPATCH_ROOT}/examples/nodejs6/secret.json
+    run dispatch create secret open-sesame -a foo-app ${DISPATCH_ROOT}/examples/nodejs/secret.json
     echo_to_log
     assert_success
 
-    run dispatch create secret open-sesame-bar -a bar-app ${DISPATCH_ROOT}/examples/nodejs6/secret.json
+    run dispatch create secret open-sesame-bar -a bar-app ${DISPATCH_ROOT}/examples/nodejs/secret.json
     echo_to_log
     assert_success
 
@@ -74,7 +74,7 @@ load variables
 
 @test "Function creation" {
 
-    run dispatch create function --image=foo-app-image foo-app-func -a foo-app ${DISPATCH_ROOT}/examples/nodejs6 --handler=./i-have-a-secret.js --secret open-sesame
+    run dispatch create function --image=foo-app-image foo-app-func -a foo-app ${DISPATCH_ROOT}/examples/nodejs --handler=./i-have-a-secret.js --secret open-sesame
     echo_to_log
     assert_success
 

e2e/tests/events.bats

@@ -13,7 +13,7 @@ load variables
     func_name=node-echo-back-${RANDOM}
     sub_name=testsub-${RANDOM}
     event_name=test.event.${RANDOM}
-    run dispatch create function --image=nodejs6 ${func_name} ${DISPATCH_ROOT}/examples/nodejs6 --handler=./debug.js
+    run dispatch create function --image=nodejs ${func_name} ${DISPATCH_ROOT}/examples/nodejs --handler=./debug.js
     echo_to_log
     assert_success
 
@@ -82,7 +82,7 @@ load variables
     sub_name=testsub-${RANDOM}
     driver_name=testdriver-${RANDOM}
 
-    run dispatch create function --image=nodejs6 ${func_name} ${DISPATCH_ROOT}/examples/nodejs6 --handler=./debug.js
+    run dispatch create function --image=nodejs ${func_name} ${DISPATCH_ROOT}/examples/nodejs --handler=./debug.js
     echo_to_log
     assert_success
 

e2e/tests/functions.bats

@@ -11,19 +11,19 @@ load variables
 
 @test "Create node function no schema" {
 
-    run dispatch create function --image=nodejs6 node-hello-no-schema ${DISPATCH_ROOT}/examples/nodejs6 --handler=./hello.js
+    run dispatch create function --image=nodejs node-hello-no-schema ${DISPATCH_ROOT}/examples/nodejs --handler=./hello.js
     echo_to_log
     assert_success
 
     run_with_retry "dispatch get function node-hello-no-schema --json | jq -r .status" "READY" 8 5
 }
 
 @test "Create a function with duplicate name" {
-    run dispatch create function --image=nodejs6 node-hello-dup ${DISPATCH_ROOT}/examples/nodejs6 --handler=./hello.js
+    run dispatch create function --image=nodejs node-hello-dup ${DISPATCH_ROOT}/examples/nodejs --handler=./hello.js
     echo_to_log
     assert_success
 
-    run dispatch create function --image=nodejs6 node-hello-dup ${DISPATCH_ROOT}/examples/nodejs6 --handler=./hello.js
+    run dispatch create function --image=nodejs node-hello-dup ${DISPATCH_ROOT}/examples/nodejs --handler=./hello.js
     assert_failure
 }
 
@@ -137,7 +137,7 @@ load variables
 }
 
 @test "Create node function with schema" {
-    run dispatch create function --image=nodejs6 node-hello-with-schema ${DISPATCH_ROOT}/examples/nodejs6 --handler=./hello.js --schema-in ${DISPATCH_ROOT}/examples/nodejs6/hello.schema.in.json --schema-out ${DISPATCH_ROOT}/examples/nodejs6/hello.schema.out.json
+    run dispatch create function --image=nodejs node-hello-with-schema ${DISPATCH_ROOT}/examples/nodejs --handler=./hello.js --schema-in ${DISPATCH_ROOT}/examples/nodejs/hello.schema.in.json --schema-out ${DISPATCH_ROOT}/examples/nodejs/hello.schema.out.json
     echo_to_log
     assert_success
 

e2e/tests/image_not_found_update.yaml

@@ -1,5 +1,5 @@
 # Test when a base image does not exist
-dockerUrl: dispatchframework/python3-base:0.0.3
+dockerUrl: dispatchframework/python3-base:0.0.7
 groups: 
 kind: BaseImage
 language: python3

e2e/tests/images.bats

@@ -10,13 +10,13 @@ load variables
     run dispatch get base-image
     assert_success
 
-    # Create base image "base-nodejs6"
-    run dispatch create base-image base-nodejs6 $DOCKER_REGISTRY/$BASE_IMAGE_NODEJS6 --language nodejs6
+    # Create base image "base-nodejs"
+    run dispatch create base-image base-nodejs $DOCKER_REGISTRY/$BASE_IMAGE_NODEJS6 --language nodejs
     assert_success
 
     # Ensure starting status is "INITIALIZED". Wait 20 seconds for status "READY"
-    run_with_retry "dispatch get base-image base-nodejs6 --json | jq -r .status" "INITIALIZED" 1 0
-    run_with_retry "dispatch get base-image base-nodejs6 --json | jq -r .status" "READY" 4 5
+    run_with_retry "dispatch get base-image base-nodejs --json | jq -r .status" "INITIALIZED" 1 0
+    run_with_retry "dispatch get base-image base-nodejs --json | jq -r .status" "READY" 4 5
 
     # Create base image "base-python3"
     run dispatch create base-image base-python3 $DOCKER_REGISTRY/$BASE_IMAGE_PYTHON3 --language python3
@@ -41,23 +41,23 @@ load variables
     run_with_retry "dispatch get base-image base-java --json | jq -r .status" "READY" 10 5
 
     # Create fifth image with non-existing image. Check that get operation returns five images. Wait for "ERROR" status for missing image.
-    run dispatch create base-image missing-image missing/image:latest --language nodejs6
+    run dispatch create base-image missing-image missing/image:latest --language nodejs
     assert_success
     run bash -c "dispatch get base-image --json | jq '. | length'"
     assert_equal 5 $output
     run_with_retry "dispatch get base-image missing-image --json | jq -r .status" "ERROR" 4 5
 }
 
 @test "Image creation" {
-    run dispatch create image nodejs6 base-nodejs6
+    run dispatch create image nodejs base-nodejs
     assert_success
     run dispatch create image python3 base-python3
     assert_success
     run dispatch create image powershell base-powershell
     assert_success
     run dispatch create image java base-java
     assert_success
-    run_with_retry "dispatch get image nodejs6 --json | jq -r .status" "READY" 8 5
+    run_with_retry "dispatch get image nodejs --json | jq -r .status" "READY" 8 5
     run_with_retry "dispatch get image python3 --json | jq -r .status" "READY" 8 5
     run_with_retry "dispatch get image powershell --json | jq -r .status" "READY" 8 5
     run_with_retry "dispatch get image java --json | jq -r .status" "READY" 8 5
@@ -90,9 +90,9 @@ load variables
     assert_success
 
     run_with_retry "dispatch get image python3 --json | jq -r .status" "READY" 6 30
-    run_with_retry "dispatch get base-image nodejs6-base --json | jq -r .status" "READY" 6 30
+    run_with_retry "dispatch get base-image nodejs-base --json | jq -r .status" "READY" 6 30
 
-    run_with_retry "dispatch get base-image nodejs6-base --json | jq -r .language" "python3" 1 0
+    run_with_retry "dispatch get base-image nodejs-base --json | jq -r .language" "python3" 1 0
     assert_success
 
     run_with_retry "dispatch get image python3 --json | jq -r .tags[0].key" "update" 1 0