Merge pull request #22 from IBMStreams/develop

Prepare new release

Author: joergboe

CHANGELOG.md

@@ -6,3 +6,25 @@
 ## v3.0.0:
 * Initial version forked from [streams.inet v2.9.6](https://github.com/IBMStreams/streamsx.inet/tree/v2.9.6)
 
+## v4.0.0
+* Remove target 'setcommitversion' from build.xml - we have now a build.info
+* Update jetty server libs to 9.4.18.v20190429
+* Sample HTTPTupleInjectAndView is now enabled for secure connections
+* Operator HTTPRequestProcess: Corrected and competed parameters and attribute
+* Operator HTTPRequestProcess: code cleanup, introduce exception handling in process method and return error responses in such cases
+* Operator HTTPRequestProcess: remove parameter statusMessage
+* Operator HTTPRequestProcess: Use Async Context instead of deprecated jetty continuation
+* Operator HTTPRequestProcess: renaming input attribute names to response...
+* Operator HTTPRequestProcess: Ignore WindowMarker and set default for non supplied json response artifacts
+* Operator HTTPRequestProcess: Add contextPath attribute to tuple output
+* Websocket Operators: Switch implementation from Java-WebSocket to jetty websocket libs
+* Fix metrics in HTTPRequestProcess
+* Improved documentation
+* Samples: Makefiles are now enabled for studio build
+* Move junit tests to tests directory
+* Integrate junit tests and framework tests into main build.xml test targets
+* New Junit Testcases for websocket operators, XMLView, PostBLOB and PostXML
+* New Framework Test Cases with HTTPRequestProcess operator attribute checks and runtime tests
+* New sample: HTTPXMLView
+* Simplified sample RequestProcessTuple
+* Removed puzzeling RequestProcessTuple Json Sample

DEVELOPMENT.md

@@ -45,10 +45,12 @@ The top-level build.xml contains the main targets:
 
 * **all** - Builds and creates SPLDOC for the toolkit and samples. Developers should ensure this target is successful when creating a pull request.
 * **toolkit** - Build the complete toolkit code
-* **samples-build** - Builds all samples. Developers should ensure this target is successful when creating a pull request.
+* **samples** - Builds all samples. Developers should ensure this target is successful when creating a pull request.
 * **release** - Builds release artifacts, which is a tar bundle containing the toolkits and samples. It includes stamping the SPLDOC and toolkit version numbers with the git commit number (thus requires git to be available).
-* **test** - Start the test
+* **test** - Start the quick test
+* **test-full** - Start all tests
 * **spldoc** - Generate the toolkit documentation
+* **clean** - Clean up toolkit code, samples code, all spldocs and downloaded libs
 
 Execute the comman `ant -p` to display the target information.
 

GRADUATION_STATUS.md

@@ -12,14 +12,14 @@ Incubation for projects is covered here: https://github.com/IBMStreams/administr
   * Status: SPLDOC complete.
 - [ ] :white_check_mark: Project has samples and instructions to help users get started easily
   * Status: Multiple samples exist.
-- [ ] :large_orange_diamond: Sufficient testing
-  * Status: Some testing exists, not all operators are tested.
+- [ ] :white_check_mark: Sufficient testing
+  * Status: All operators are tested.
 - [ ] :white_check_mark: Instructions on how to get started with development and contributing to project
   * Status: Description exists
 - [ ] :white_check_mark: Website that contains information and documentation of the project
   * Status: http://ibmstreams.github.io/streamsx.inet/
-- [ ] :red_circle: Clear release plan
-  * Status: **TBD**
+- [ ] :white_check_mark: Clear release plan
+  * Status: Clear release plan exists
 - [ ] :white_check_mark: At least one active committer
   * Status: There are more than one active commiters
 

README.md

@@ -1,7 +1,7 @@
 # streamsx.inetserver
 
-The INet Server Toolkit
-The Internet Toolkit provides support for common internet rest protocol and web socket server protocol.
+The Internet Server Toolkit provides support HTTP- and Websocket-Server functions. The functionality is based on an embedded 
+jetty webserver. It may be used for rest protocol and web socket protocol applications.
 This toolkit is a split from streamsx.inet 2.9.6
 [streams.inet v2.9.6](https://github.com/IBMStreams/streamsx.inet/tree/v2.9.6)
 

build.xml

@@ -5,14 +5,16 @@
 	<!-- set global properties for this build -->
 	<property environment="env" />
 	<property name="streams.install"     value="${env.STREAMS_INSTALL}" />
-	<property name="toolkit.dir"         value="com.ibm.streamsx.inetserver"/>
+	<property name="toolkit.string"      value="com.ibm.streamsx.inetserver"/>
+	<property name="toolkit.dir"         location="${toolkit.string}"/> <!-- must be absolute because is used in called test scripts -->
 	<property name="doc.string"          value="doc"/>
 	<property name="doc.dir"             value="${doc.string}/spldoc"/>
 	<property name="samples.dir"         value="samples"/>
 	<property name="samples.doc.dir"     value="samples/${doc.string}/spldoc"/>
 	<property name="test.dir"            value="tests"/>
+	<property name="junit.test.dir"      value="${test.dir}/junit"/>
 	<property name="release.dir"         value="release" />
-	<property name="release.info.file"   value="${toolkit.dir}/build.info"/>
+	<property name="release.info.file"   value="${toolkit.string}/build.info"/>
 
 	<property name="tktitle"             value="IBMStreams ${ant.project.name} Toolkit" />
 	<property name="tkauthor"            value="IBMStreams Open Source Community at GitHub - https://github.com/IBMStreams/${ant.project.name}" />
@@ -28,7 +30,7 @@
 		description="Main target: Build the toolkit code, build the samples index and generate the spl documentation - incremental build"
 	/>
 
-	<target name="clean" depends="spldoc-clean,samples-clean,releaseinfo-clean"
+	<target name="clean" depends="spldoc-clean,samples-clean,releaseinfo-clean,test-clean"
 		description="Main target: clean up toolkit code, samples code, all spldocs and downloaded libs">
 		<ant dir="${toolkit.dir}" target="clean" />
 	</target>
@@ -44,7 +46,7 @@
 	</target>
 
 	<target name="get_name">
-		<xmlproperty file="${toolkit.dir}/info.xml" prefix="tkinfo" keepRoot="no"/>
+		<xmlproperty file="${toolkit.string}/info.xml" prefix="tkinfo" keepRoot="no"/>
 		<echo message="Toolkit Name: ${tkinfo.identity.name}"/>
 		<echo message="Toolkit Version: ${tkinfo.identity.version}"/>
 	</target>
@@ -59,7 +61,7 @@
 			<arg value="--doc-title" />
 			<arg value="${tktitle}" />
 			<arg value="--directory" />
-			<arg value="${toolkit.dir}" />
+			<arg value="${toolkit.string}" />
 			<arg value="--warn-no-comments"/>
 			<arg value="--copy-image-files" />
 			<arg value="--output-directory" />
@@ -128,37 +130,38 @@
 	</target>
 
 	<!-- Test targets -->
-	<target name="test" depends="toolkit"
-		description="Execute the toolkit test. Requires an up to date toolkit build">
-		<echo message="Start full test"/>
-		<exec executable="./runTest.sh" dir="${test.dir}/frameworktests" failonerror="true">
-			<arg value="--noprompt"/>
-			<arg value="--no-browser"/>
+	<target name="test" depends="test-junit"
+		description="Main target: Execute the toolkit quick test. Includes junit tests.">
+		<echo message="Start quick test"/>
+		<exec executable="make" dir="${test.dir}/frameworktests" failonerror="true">
+			<arg value="test"/>
 		</exec>
 	</target>
 
-	<target name="test-quick" depends="toolkit"
-		description="Execute the toolkit quick test. Requires an up to date toolkit build">
-		<echo message="Start quick test"/>
-		<exec executable="./runTest.sh" dir="${test.dir}/frameworktests" failonerror="true">
-			<arg value="--noprompt"/>
-			<arg value="--no-browser"/>
-			<arg value="--category"/>
-			<arg value="quick"/>
+	<target name="test-full" depends="test-junit"
+		description="Main target: Execute all toolkit tests. Includes junit tests.">
+		<echo message="Start full test"/>
+		<exec executable="make" dir="${test.dir}/frameworktests" failonerror="true">
+			<arg value="test-full"/>
 		</exec>
 	</target>
 
+	<target name="test-junit" depends="toolkit"
+		description="Execute the junit tests">
+		<echo message="Execute the junit tests"/>
+		<ant dir="${junit.test.dir}" target="test"/>
+	</target>
+
 	<target name="test-clean"
 		description="Clean up all generated test artifacts">
+		<ant dir="${junit.test.dir}" target="clean-all"/>
+		<exec executable="make" dir="${test.dir}/frameworktests" failonerror="true">
+			<arg value="clean-all"/>
+		</exec>
 		<delete dir="${test.dir}/frameworktests/workdir"/>
 		<delete dir="${test.dir}/frameworktests/scripts"/>
 	</target>
 
-	<target name="tests" depends="toolkit">
-		<ant dir="${toolkit.dir}" target="test"/>
-		<ant dir="${toolkit.test}" target="all"/>
-	</target>
-
 	<!-- Targets to build releases -->
 	<target name="release" depends="clean"
 		description="Make a toolkit release archive - purge workspace and build toolkit from scratch">
@@ -183,8 +186,8 @@
 		<tar compression="gzip" longfile="gnu"
 					destfile="${releasefilename}"
 					basedir="${basedir}"
-					includes="${toolkit.dir}/** samples/** ${doc.string}/** README.md LICENSE"
-					excludes="**/.toolkitList **/.gitignore **/.gitkeep ${toolkit.dir}/.project ${toolkit.dir}/.settings/ ${toolkit.dir}/.classpath ${toolkit.dir}/impl/java/ ${toolkit.dir}/icons/ ${toolkit.dir}/build.xml ${toolkit.dir}/pom.xml"
+					includes="${toolkit.string}/** samples/** ${doc.string}/** README.md LICENSE"
+					excludes="**/.toolkitList **/.gitignore **/.gitkeep ${toolkit.string}/.project ${toolkit.string}/.settings/ ${toolkit.string}/.classpath ${toolkit.string}/impl/java/ ${toolkit.string}/icons/ ${toolkit.string}/build.xml ${toolkit.string}/pom.xml"
 		/>
 		<checksum file="${releasefilename}"/>
 		<checksum algorithm="sha1" file="${releasefilename}"/>

com.ibm.streamsx.inetserver/.gitignore

@@ -2,5 +2,4 @@
 /impl/java/src-gen/
 /impl/java/studio-build/
 /output/
-/unittestrun/
 /lib/

com.ibm.streamsx.inetserver/build.xml

@@ -27,8 +27,6 @@ artifacts were left around and caused issues with the ant build.
 	<property name="gensrc.dir2"    location="impl/java/src-gen2"/>     <!-- streams studio bin dir / used only on clean target -->
 	<property name="out.dir"        location="output"/>                 <!-- streams studio bin dir / used only on clean target -->
 
-	<property name="test.run.dir"   location="unittestrun" />
-	<property name="test.build.dir" location="${test.run.dir}/bin" />
 	<property name="downloads.dir"  location="opt/downloaded" /> <!-- used for cleanup only -->
 	<property name="jarfile"        value="com.ibm.streamsx.inetserver.jar" />
 
@@ -87,8 +85,8 @@ artifacts were left around and caused issues with the ant build.
 		<delete dir="${gensrc.dir}"/>
 	</target>
 
-	<target name="clean" depends="toolkit-clean, test-clean"
-		description="Clean up generated toolkit artifacts and downloaded libraries and test artifacts">
+	<target name="clean" depends="toolkit-clean"
+		description="Clean up generated toolkit artifacts and downloaded libraries">
 		<delete dir="${lib.dir}"/>
 		<delete dir="${downloads.dir}" />
 		<delete dir="doc"/> <!-- delete old doc files -->
@@ -127,8 +125,7 @@ artifacts were left around and caused issues with the ant build.
 	<target name="compile" depends="maven-deps"
 		description="Compile the java source components">
 		<javac srcdir="${src.dir}" destdir="${build.dir}" debug="true" includeantruntime="no"
-			source="1.8" target="1.8"
-			excludes="com/ibm/streamsx/inet/*/test/**">
+			source="1.8" target="1.8">
 			<classpath>
 				<path refid="cp.compile" />
 			</classpath>
@@ -152,45 +149,5 @@ artifacts were left around and caused issues with the ant build.
 		<!-- <delete dir="${build.dir}" /> do not delete classes to enable incremental build -->
 	</target>
 
-	<!-- assumes Junit 4.3 (or later) is in $HOME/.ant/lib -->
-	<target name="test" depends="jar"
-		description="Execute the integrated unit tests">
-		<mkdir dir="${test.run.dir}" />
-		<mkdir dir="${test.build.dir}" />
-
-		<javac srcdir="${src.dir}" destdir="${test.build.dir}" debug="true" includeantruntime="yes" includes="com/ibm/streamsx/inet/*/test/**">
-			<classpath>
-				<pathelement location="${impl.lib.dir}/${jarfile}" />
-				<path refid="cp.streams" />
-			</classpath>
-		</javac>
-
-		<junit printsummary="yes" haltonfailure="yes" fork="yes">
-			<classpath>
-				<pathelement location="${test.build.dir}" />
-				<pathelement location="${impl.lib.dir}/${jarfile}" />
-				<path refid="cp.streams" />
-			</classpath>
-
-			<formatter type="xml" />
-
-			<batchtest todir="${test.run.dir}">
-				<fileset dir="${src.dir}">
-					<include name="**/*Test.java" />
-				</fileset>
-			</batchtest>
-		</junit>
-		<!-- <junitreport todir="${test.run.dir}">
-			<fileset dir="${test.run.dir}">
-				<include name="TEST-*.xml"/>
-			</fileset>
-			<report format="frames" todir="${test.run.dir}/html"/>
-		</junitreport> -->
-	</target>
-
-	<target name="test-clean" description="Clean generated test artifacts">
-		<delete dir="${test.run.dir}" />
-		<delete dir="${test.build.dir}" />
-	</target>
 </project>
 

com.ibm.streamsx.inetserver/com.ibm.streamsx.inet/namespace-info.spl

@@ -5,7 +5,7 @@
  * + Sharing the Eclipse Jetty Instance
  * Multiple operators share a single jetty server instance if:
  * * they are fused in a single processing element (config placement: partitionColocation),
- * * and they use the same `port` parameter
+ * * and they use the same `port` parameter value
  * This allows operators to expose streams through the same webserver
  * without having to be connected in the SPL graph.
  * 

com.ibm.streamsx.inetserver/impl/java/src/com/ibm/streamsx/inet/rest/ops/PostJSON.java

@@ -45,7 +45,7 @@ public static void checkJSONAttribute(OperatorContextChecker checker) {
 				checker.checkAttributeType(jsonString, MetaType.RSTRING);
 				continue;
 			}
-							
+			
 			Attribute first = port.getStreamSchema().getAttribute(0);
 			checker.checkAttributeType(first, MetaType.RSTRING);
 		}