[Info-vax] openvms and xterm
Arne Vajhøj
arne at vajhoej.dk
Wed Apr 24 20:27:56 EDT 2024
On 4/24/2024 8:08 PM, Arne Vajhøj wrote:
> This will be a relative long post. Sorry.
And this one will be even longer!
> The problem at hand has nothing to do with JSON. It is
> a string to numeric and data types problem.
>
> JSON:
>
> { "v": 100000000000000001 }
>
> XML:
>
> <data>
> <v>100000000000000001</v>
> </data>
>
> YAML:
>
> v: 100000000000000001
>
> All expose the same problem.
>
> The value cannot be represented as is in some very common
> data types like 32 bit integers and 64 bit floating point.
> The fact that it ultimately is the developers responsibility
> to select proper data types does not mean that programming languages
> and JSON libraries can not help catch errors.
>
> If it is obvious that an unexpected/useless result is being
> produced then it should be flagged (return error code or throw
> exception depending on technology).
>
> Let us go back to the example with 100000000000000001.
>
> Trying to stuff that into a 32 bit integer by like parsing
> it as a 64 bit integer and returning the lower 32 bits
> is in my best opinion an error. Nobody wants to get an int
> with 1569325057 from retrieving a 32 bit integer integer
> from "100000000000000001". It should give an error.
>
> The case with a 64 bit floating point is more tricky. One
> could argue that 100000000000000001.0 is the expected
> result and that 100000000000000000.0 should be considered
> an error. And it probably would be an error in the majority
> of cases. But there is actually the possibility that
> someone that understand floating point are reading JSON
> and expect what is happening and does not care because
> there are some uncertainty in the underlying data. And
> creating a false error for people that understand FP data
> types to prevent those that do not understand FP data types
> from shooting themself in the foot is not good.
Let us see some code.
I picked Groovy as demo language, because I am a J guy
and Groovy allows to demo a lot.
As a JVM language there are several possible data types to
pick from:
int
long
double
String
BigInteger
BigDecimal
And obviously int and double has problem with 100000000000000001
while the rest can store it OK.
To illustrate the option of different JSON libraries I will
test with both GSON (Google JSON library) and Jackson (probably
the most widely used JSON library in the Java wold).
Let us first look at the model where the JSON is parsed to
a tree.
GSON:
$ type Data1.groovy
import com.google.gson.*
def wrap(block) {
try {
block()
} catch(ex) {
printf("%s: %s\n", ex.class.name, ex.message)
}
}
String json = '{ "v": 100000000000000001 }'
println(json)
gson = new JsonParser()
o = gson.parse(json).getAsJsonObject()
wrap {
v1 = o.get("v").getAsInt()
printf("int: %d\n", v1)
}
wrap {
v2 = o.get("v").getAsLong()
printf("long: %d (low 32 bit is %d)\n", v2, v2 & 0x00000000FFFFFFFFL)
}
wrap {
v3 = o.get("v").getAsDouble()
printf("double: %f\n", v3)
}
wrap {
v4 = o.get("v").getAsString()
printf("String: %s\n", v4)
}
wrap {
v5 = o.get("v").getAsBigInteger()
printf("BigInteger: %s\n", v5)
}
wrap {
v6 = o.get("v").getAsBigDecimal()
printf("BigDecimal: %s\n", v6)
}
wrap {
v7 = o.get("v").getAsNumber()
printf("Number: %s (%s)\n", v7, v7.class.name)
}
$ groovy Data1.groovy
{ "v": 100000000000000001 }
int: 1569325057
long: 100000000000000001 (low 32 bit is 1569325057)
double: 100000000000000000.000000
String: 100000000000000001
BigInteger: 100000000000000001
BigDecimal: 100000000000000001
Number: 100000000000000001 (com.google.gson.internal.LazilyParsedNumber)
Jackson:
$ type Data1X.groovy
import com.fasterxml.jackson.databind.*
def wrap(block) {
try {
block()
} catch(ex) {
printf("%s: %s\n", ex.class.name, ex.message)
}
}
String json = '{ "v": 100000000000000001 }'
println(json)
om = new ObjectMapper()
o = om.readTree(json)
wrap {
v1 = o.get("v").asInt()
printf("int: %d\n", v1)
}
wrap {
v2 = o.get("v").asLong()
printf("long: %d (low 32 bit is %d)\n", v2, v2 & 0x00000000FFFFFFFFL)
}
wrap {
v3 = o.get("v").asDouble()
printf("double: %f\n", v3)
}
wrap {
v4 = o.get("v").asText()
printf("String: %s\n", v4)
}
wrap {
v5 = o.get("v").bigIntegerValue()
printf("BigInteger: %s\n", v5)
}
wrap {
v6 = o.get("v").decimalValue()
printf("BigDecimal: %s\n", v6)
}
wrap {
v7 = o.get("v").numberValue()
printf("Number: %s (%s)\n", v7, v7.class.name)
}
$ groovy Data1X.groovy
{ "v": 100000000000000001 }
int: 1569325057
long: 100000000000000001 (low 32 bit is 1569325057)
double: 100000000000000000.000000
String: 100000000000000001
BigInteger: 100000000000000001
BigDecimal: 100000000000000001
Number: 100000000000000001 (java.lang.Long)
We see:
* the expected slightly off value for double
* the crazy value for int (no exception)
* other data types are fine
Now mapping/binding to class.
GSON:
$ type Data2.groovy
import com.google.gson.*
class C1 {
int v
}
class C2 {
long v
}
class C3 {
double v
}
class C4 {
String v
}
class C5 {
BigInteger v
}
class C6 {
BigDecimal v
}
class C7 {
Number v
}
def wrap(block) {
try {
block()
} catch(ex) {
printf("%s: %s\n", ex.class.name, ex.message)
}
}
json = '{ "v": 100000000000000001 }'
println(json)
gson = new Gson()
wrap {
C1 v1 = gson.fromJson(json, C1.class)
printf("int: %d\n", v1.v)
}
wrap {
C2 v2 = gson.fromJson(json, C2.class)
printf("long: %d\n", v2.v)
}
wrap {
C3 v3 = gson.fromJson(json, C3.class)
printf("double: %f\n", v3.v)
}
wrap {
C4 v4 = gson.fromJson(json, C4.class)
printf("String: %s\n", v4.v)
}
wrap {
C5 v5 = gson.fromJson(json, C5.class)
printf("BigInteger: %s\n", v5.v)
}
wrap {
C6 v6 = gson.fromJson(json, C6.class)
printf("BigDecimal: %s\n", v6.v)
}
wrap {
C7 v7 = gson.fromJson(json, C7.class)
printf("Number: %s (%s)\n", v7.v, v7.v.class.name)
}
$ groovy Data2.groovy
{ "v": 100000000000000001 }
com.google.gson.JsonSyntaxException: java.lang.NumberFormatException:
Expected an int but was 100000000000000001 at line 1 column 26
long: 100000000000000001
double: 100000000000000000.000000
String: 100000000000000001
BigInteger: 100000000000000001
BigDecimal: 100000000000000001
Number: 100000000000000001 (com.google.gson.internal.LazilyParsedNumber)
Jackson:
$ type Data2X.groovy
import com.fasterxml.jackson.databind.*
class C1 {
int v
}
class C2 {
long v
}
class C3 {
double v
}
class C4 {
String v
}
class C5 {
BigInteger v
}
class C6 {
BigDecimal v
}
class C7 {
Number v
}
def wrap(block) {
try {
block()
} catch(ex) {
printf("%s: %s\n", ex.class.name, ex.message)
}
}
json = '{ "v": 100000000000000001 }'
println(json)
om = new ObjectMapper()
wrap {
C1 v1 = om.readValue(json, C1.class)
printf("int: %d\n", v1.v)
}
wrap {
C2 v2 = om.readValue(json, C2.class)
printf("long: %d\n", v2.v)
}
wrap {
C3 v3 = om.readValue(json, C3.class)
printf("double: %f\n", v3.v)
}
wrap {
C4 v4 = om.readValue(json, C4.class)
printf("String: %s\n", v4.v)
}
wrap {
C5 v5 = om.readValue(json, C5.class)
printf("BigInteger: %s\n", v5.v)
}
wrap {
C6 v6 = om.readValue(json, C6.class)
printf("BigDecimal: %s\n", v6.v)
}
wrap {
C7 v7 = om.readValue(json, C7.class)
printf("Number: %s (%s)\n", v7.v, v7.v.class.name)
}
$ groovy Data2X.groovy
{ "v": 100000000000000001 }
com.fasterxml.jackson.databind.JsonMappingException: Numeric value
(100000000000000001) out of range of int (-2147483648 - 2147483647)
at [Source: (String)"{ "v": 100000000000000001 }"; line: 1, column:
26] (through reference chain: C1["v"])
long: 100000000000000001
double: 100000000000000000.000000
String: 100000000000000001
BigInteger: 100000000000000001
BigDecimal: 100000000000000001
Number: 100000000000000001 (java.lang.Long)
Very similar behavior except that now I do get the exception for
int that I so much prefer.
Long post. But hey the above is run on VMS 9.2-2!
Arne
More information about the Info-vax
mailing list