Skip to content

TINKERPOP-3166 Implement asNumber() step #3153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: 3.8-dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ This release also includes changes from <<release-3-7-XXX, 3.7.XXX>>.
* Moved all lambda oriented Gremlin tests to `LambdaStepTest` in the Java test suite.
* Removed the `@RemoteOnly` testing tag in Gherkin as lambda tests have all been moved to the Java test suite.
* Updated gremlin-javascript to use GraphBinary as default instead of GraphSONv3
* Added the `asNumber()` step to perform number conversion.
* Renamed many types in the grammar for consistent use of terms "Literal", "Argument", and "Varargs"
* Changed `gremlin-net` so that System.Text.Json is only listed as an explicit dependency when it is not available from the framework.

Expand Down
27 changes: 27 additions & 0 deletions docs/src/dev/provider/gremlin-semantics.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -827,6 +827,33 @@ Incoming date remains unchanged.
See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsDateStep.java[source],
link:https://tinkerpop.apache.org/docs/x.y.z/reference/#asDate-step[reference]

[[asNumber-step]]
=== asNumber()

*Description:* converts the incoming traverser to the nearest parsable type if no argument is provided, or to the desired numerical type, based on the number token (`N`) provided.

*Syntax:* `asNumber()` | `asNumber(N numberToken)`

[width="100%",options="header"]
|=========================================================
|Start Step |Mid Step |Modulated |Domain |Range
|N |Y |N |`Number`/`String` |`Number`
|=========================================================

*Arguments:*

* `numberToken` - The enum `N` to denote the desired type to parse/cast to.

If no type token is provided, the incoming number remains unchanged.

*Exceptions*
* If any overflow occurs during narrowing of types, then an `ArithmeticException` will be thrown.
* If the incoming string cannot be parsed into a valid number format, then a `NumberFormatException` will be thrown.
* If the incoming traverser is a non-String/Number (including `null`) value then an `IllegalArgumentException` will be thrown.

See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java[source],
link:https://tinkerpop.apache.org/docs/x.y.z/reference/#asNumber-step[reference]

[[barrier-step]]
=== barrier()

Expand Down
46 changes: 46 additions & 0 deletions docs/src/reference/the-traversal.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -850,6 +850,52 @@ g.inject(datetime("2023-08-24T00:00:00Z")).asDate() <3>

link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#asDate()++[`asDate()`]

[[asNumber-step]]
=== AsNumber Step

The `asNumber()`-step (*map*) converts the incoming traverser to the nearest parsable type if no argument is provided, or to the desired numerical type, based on the number token (`N`) provided.

Numerical input will pass through unless a type is specified by the number token. `ArithmeticException` will be thrown for any overflow during narrowing of types.

String inputs are parsed into numeric values. By default, the value will be parsed as an integer if it represents a whole number, or as a double if it contains a decimal point. A `NumberFormatException` will be thrown if the string cannot be parsed into a valid number format.

All other input types will result in `IllegalArgumentException`.

[gremlin-groovy,modern]
----
g.inject(1234).asNumber() <1>
g.inject(1.76).asNumber() <2>
g.inject(1.76).asNumber(N.int_) <3>
g.inject("1b").asNumber() <4>
g.inject(33550336).asNumber(N.byte_) <5>
----

<1> An int will be passed through.
<2> A double will be passed through.
<3> A double is converted into an int.
<4> String containing any character other than numerical ones will result in `NumberFormatException`.
<5> Narrowing of int to byte that overflows will throw `ArithmeticException`.

[NOTE, caption=Java]
====
The enums values `byte`, `short`, `int`, `long`, `float`, `double` are reserved word in Java, and therefore must be referred to in Gremlin with an underscore appended as a suffix: `byte_`, `short_`, `int_`, `long_`, `float_`, `double_`.
====

[NOTE, caption=Groovy & Gremlin Console]
====
The enums values `byte`, `short`, `int`, `long`, `float`, `double` are reserved word in Groovy, therefore as the Gremlin Console is Groovy-based, they must be referred to in Gremlin with an underscore appended as a suffix: `byte_`, `short_`, `int_`, `long_`, `float_`, `double_`.
====

[NOTE, caption=JavaScript]
====
The enums values `byte`, `short`, `int`, `long`, `float`, `double` are reserved word in Javascript, and therefore must be referred to in Gremlin with an underscore appended as a suffix: `byte_`, `short_`, `int_`, `long_`, `float_`, `double_`.
====

*Additional References*

link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#asNumber()++[`asNumber()`]
link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#asNumber(org.apache.tinkerpop.gremlin.process.traversal.N)++[`asNumber(N)`]

[[barrier-step]]
=== Barrier Step

Expand Down
63 changes: 63 additions & 0 deletions docs/src/upgrade/release-3.8.x.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,69 @@ complete list of all the modifications that are part of this release.

=== Upgrading for Users

==== Number Conversion Step

We have been iteratively introducing new language features into Gremlin, with the most recent major additions being string, list and date manipulation
steps introduced in the 3.7 line. In 3.8.0, we are now introducing a number conversion step, `asNumber()`, to bridge
a gap in casting functionalities.

The new `asNumber()` serves as an umbrella step that parses strings and casts numbers into desired types. For the convenience of remote traversals in GLVs, these number types are denoted by a set of number tokens (`N`).

This new step will allow users to normalize their data by converting string numbers and mixed numeric types to a consistent format, making it easier to perform downstream mathematical operations. As an example:

[source,text]
----
// sum() step can only take numbers
gremlin> g.inject(1.0, 2l, 3, "4", "0x5").sum()
class java.lang.String cannot be cast to class java.lang.Number

// use asNumber() to avoid casting exceptions
gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum()
==>15.0

// given sum() step returned a double, one can use asNumber() to further cast the result into desired type
gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum().asNumber(N.int_)
==>15
----

Semantically, the `asNumber()` step will convert the incoming traverser to a logical parsable type if no argument is provided, or to the desired numerical type, based on the number token (`N`) provided.

Numerical input will pass through unless a type is specified by the number token. `ArithmeticException` will be thrown for any overflow as a result of narrowing of types:

[source,text]
----
gremlin> g.inject(5.0).asNumber(N.int_)
==> 5 // casts double to int
gremlin> g.inject(12).asNumber(N.byte_)
==> 12
gremlin> g.inject(128).asNumber(N.byte_)
==> ArithmeticException
----

String input will be parsed. By default, the smalled unit of number to be parsed into is `int` if no number token is provided. `NumberFormatException` will be thrown for any unparsable strings:

[source,text]
----
gremlin> g.inject("5").asNumber()
==> 5
gremlin> g.inject("5.7").asNumber(N.int_)
==> 5
gremlin> g.inject("1,000").asNumber(N.int_)
==> NumberFormatException
gremlin> g.inject("128").asNumber(N.byte_)
==> ArithmeticException
----

All other input types will result in `IllegalArgumentException`:
[source,text]
----
gremlin> g.inject([1, 2, 3, 4]).asNumber()
==> IllegalArgumentException
----

See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#asNumber-step[asNumber()-step]
See: link:https://issues.apache.org/jira/browse/TINKERPOP-3166[TINKERPOP-3166]

==== Boolean Conversion Step

The `asBool()` step bridges another gap in Gremlin's casting functionalities. Users now have the ability to parse strings and
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.IO;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Operator;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.P;
Expand Down Expand Up @@ -204,6 +205,7 @@ public final class CoreImports {
CLASS_IMPORTS.add(Direction.class);
CLASS_IMPORTS.add(DT.class);
CLASS_IMPORTS.add(Merge.class);
CLASS_IMPORTS.add(N.class);
CLASS_IMPORTS.add(Operator.class);
CLASS_IMPORTS.add(Order.class);
CLASS_IMPORTS.add(Pop.class);
Expand Down Expand Up @@ -363,6 +365,7 @@ public final class CoreImports {
Collections.addAll(ENUM_IMPORTS, Direction.values());
Collections.addAll(ENUM_IMPORTS, DT.values());
Collections.addAll(ENUM_IMPORTS, Merge.values());
Collections.addAll(ENUM_IMPORTS, N.values());
Collections.addAll(ENUM_IMPORTS, Operator.values());
Collections.addAll(ENUM_IMPORTS, Order.values());
Collections.addAll(ENUM_IMPORTS, Pop.values());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1071,6 +1071,14 @@ protected void notImplemented(final ParseTree ctx) {
* {@inheritDoc}
*/
@Override public T visitTraversalMethod_dateDiff_Date(final GremlinParser.TraversalMethod_dateDiff_DateContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
@Override public T visitTraversalMethod_asNumber_Empty(final GremlinParser.TraversalMethod_asNumber_EmptyContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
@Override public T visitTraversalMethod_asNumber_traversalN(final GremlinParser.TraversalMethod_asNumber_traversalNContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
Expand Down Expand Up @@ -1111,6 +1119,10 @@ protected void notImplemented(final ParseTree ctx) {
* {@inheritDoc}
*/
@Override public T visitTraversalDT(GremlinParser.TraversalDTContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
@Override public T visitTraversalN(GremlinParser.TraversalNContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -601,6 +601,11 @@ public Object visitTraversalPick(final GremlinParser.TraversalPickContext ctx) {
return TraversalEnumParser.parseTraversalEnumFromContext(Pick.class, ctx);
}

@Override
public Object visitTraversalN(final GremlinParser.TraversalNContext ctx) {
return TraversalEnumParser.parseTraversalNFromContext(ctx);
}

@Override
public Object visitTraversalStrategy(final GremlinParser.TraversalStrategyContext ctx) {
return antlr.traversalStrategyVisitor.visitTraversalStrategy(ctx);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
package org.apache.tinkerpop.gremlin.language.grammar;

import org.antlr.v4.runtime.tree.ParseTree;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Scope;
import org.apache.tinkerpop.gremlin.structure.Direction;

Expand Down Expand Up @@ -65,4 +66,14 @@ public static Direction parseTraversalDirectionFromContext(final GremlinParser.T
text = text.substring(Direction.class.getSimpleName().length() + 1);
return Direction.directionValueOf(text);
}

/**
* Parsing of {@link N} requires some special handling because of java keyword collision.
*/
public static N parseTraversalNFromContext(final GremlinParser.TraversalNContext context) {
String text = context.getText();
if (text.startsWith(N.class.getSimpleName()))
text = text.substring(N.class.getSimpleName().length() + 1);
return text.startsWith("big") ? N.valueOf(text) : N.valueOf(text + "_");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Operator;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.Pop;
Expand Down Expand Up @@ -2110,6 +2111,22 @@ public GraphTraversal visitTraversalMethod_dateDiff_Date(final GremlinParser.Tra
return graphTraversal.dateDiff(antlr.genericVisitor.parseDate(ctx.dateLiteral()));
}

/**
* {@inheritDoc}
*/
@Override
public GraphTraversal visitTraversalMethod_asNumber_Empty(final GremlinParser.TraversalMethod_asNumber_EmptyContext ctx) {
return graphTraversal.asNumber();
}

/**
* {@inheritDoc}
*/
@Override
public GraphTraversal visitTraversalMethod_asNumber_traversalN(final GremlinParser.TraversalMethod_asNumber_traversalNContext ctx) {
return graphTraversal.asNumber(
TraversalEnumParser.parseTraversalNFromContext(ctx.traversalN()));
}

public GraphTraversal[] getNestedTraversalList(final GremlinParser.NestedTraversalListContext ctx) {
return ctx.nestedTraversalExpr().nestedTraversal()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ public GroovyTranslateVisitor(final String graphTraversalSourceName) {
super(graphTraversalSourceName);
}

@Override
public Void visitTraversalN(GremlinParser.TraversalNContext ctx) {
final String[] split = ctx.getText().split("\\.");
sb.append(processGremlinSymbol(split[0])).append(".");
sb.append(processGremlinSymbol(split[1]));
if (!split[1].startsWith("big")) sb.append("_");
return null;
}

@Override
public Void visitIntegerLiteral(final GremlinParser.IntegerLiteralContext ctx) {
final String integerLiteral = ctx.getText().toLowerCase();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.TerminalNode;
import org.apache.tinkerpop.gremlin.language.grammar.GremlinParser;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.OptionsStrategy;
import org.apache.tinkerpop.gremlin.structure.Direction;
import org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertex;
import org.apache.tinkerpop.gremlin.util.DatetimeHelper;

Expand Down Expand Up @@ -106,6 +108,15 @@ public Void visitClassType(final GremlinParser.ClassTypeContext ctx) {
return null;
}

@Override
public Void visitTraversalN(GremlinParser.TraversalNContext ctx) {
final String[] split = ctx.getText().split("\\.");
sb.append(processGremlinSymbol(split[0])).append(".");
sb.append(processGremlinSymbol(split[1]));
if (!split[1].startsWith("big")) sb.append("_");
return null;
}

@Override
public Void visitGenericMapLiteral(final GremlinParser.GenericMapLiteralContext ctx) {
sb.append("new LinkedHashMap<Object, Object>() {{ ");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,15 @@ public Void visitConfiguration(final GremlinParser.ConfigurationContext ctx) {
return null;
}

@Override
public Void visitTraversalN(GremlinParser.TraversalNContext ctx) {
final String[] split = ctx.getText().split("\\.");
sb.append(processGremlinSymbol(split[0])).append(".");
sb.append(processGremlinSymbol(split[1]));
if (!split[1].startsWith("big")) sb.append("_");
return null;
}

@Override
public Void visitGenericMapLiteral(final GremlinParser.GenericMapLiteralContext ctx) {
sb.append("new Map([");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import org.apache.tinkerpop.gremlin.language.grammar.GremlinParser;
import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Operator;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.P;
Expand Down Expand Up @@ -207,6 +208,12 @@ public Void visitTraversalDT(final GremlinParser.TraversalDTContext ctx) {
return null;
}

@Override
public Void visitTraversalN(final GremlinParser.TraversalNContext ctx) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a few cases to GremlinTranslatorTest to ensure N is translating correctly?

appendExplicitNaming(ctx.getText(), N.class.getSimpleName());
return null;
}

@Override
public Void visitTraversalPredicate(final GremlinParser.TraversalPredicateContext ctx) {
switch(ctx.getChildCount()) {
Expand Down
Loading
Loading