Truncation Error in large Systems

[matthiashimmelmann]

Hey guys!

Particularly when solving large polynomial systems via parameter homotopy, I get the following truncation error:

Computing mixed cells... 296     Time: 0:00:11
  mixed_volume:  40320
ERROR: LoadError: InexactError: trunc(Int32, 1721321924321454894)
Stacktrace:
 [1] throw_inexacterror(::Symbol, ::Type{Int32}, ::Int64) at .\boot.jl:558
 [2] checked_trunc_sint at .\boot.jl:580 [inlined]
 [3] toInt32 at .\boot.jl:617 [inlined]
 [4] Int32 at .\boot.jl:707 [inlined]
 [5] convert at .\number.jl:7 [inlined]
 [6] cconvert at .\essentials.jl:388 [inlined]
 [7] set_si! at .\gmp.jl:201 [inlined]
 [8] _broadcast_getindex_evalf at .\broadcast.jl:648 [inlined]
 [9] _broadcast_getindex at .\broadcast.jl:621 [inlined]
 [10] getindex at .\broadcast.jl:575 [inlined]
 [11] macro expansion at .\broadcast.jl:932 [inlined]
 [12] macro expansion at .\simdloop.jl:77 [inlined]
 [13] copyto! at .\broadcast.jl:931 [inlined]
 [14] copyto! at .\broadcast.jl:886 [inlined]
 [15] copy at .\broadcast.jl:862 [inlined]
 [16] materialize at .\broadcast.jl:837 [inlined]
 [17] solve!(::HomotopyContinuation.BinomialSystemSolver) at C:\Users\matth\.julia\packages\HomotopyContinuation\Mpf1C\src\binomial_system.jl:178
 [18] solve at C:\Users\matth\.julia\packages\HomotopyContinuation\Mpf1C\src\binomial_system.jl:163 [inlined]
 [19] iterate(::HomotopyContinuation.PolyhedralStartSolutionsIterator, ::Tuple{Int64,Int64}) at C:\Users\matth\.julia\packages\HomotopyContinuation\Mpf1C\src\polyhedral.jl:81
 [20] _collect(::UnitRange{Int64}, ::HomotopyContinuation.PolyhedralStartSolutionsIterator, ::Base.HasEltype, ::Base.SizeUnknown) at .\array.jl:636
 [21] collect at .\array.jl:624 [inlined]
 [22] #solve#274 at C:\Users\matth\.julia\packages\HomotopyContinuation\Mpf1C\src\solve.jl:499 [inlined]
 [23] solve(::HomotopyContinuation.ModelKit.System; show_progress::Bool, threading::Bool, catch_interrupt::Bool, target_parameters::Array{Complex{Float64},1}, stop_early_cb::Function, transform_result::Nothing, transform_parameters::typeof(identity), flatten::Nothing, target_subspaces::Nothing, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\matth\.julia\packages\HomotopyContinuation\Mpf1C\src\solve.jl:485
 [24] catastrophePoints(::Array{Array{T,1} where T,1}, ::Array{HomotopyContinuation.ModelKit.Variable,1}, ::Array{HomotopyContinuation.ModelKit.Variable,1}, ::Array{Float64,1}, ::Array{Array{Int64,1},1}, ::Array{Array{Float64,1},1}) at C:\Users\matth\equilibriaTensegrityFramework.jl\equilibriaTensegrityFramework.jl\src\functionsForStableEquilibria.jl:245
 [25] stableEquilibria(::Array{Array{T,1} where T,1}, ::Array{Array{Float64,1},1}, ::Array{Array{Int64,1},1}, ::Array{HomotopyContinuation.ModelKit.Variable,1}, ::Array{HomotopyContinuation.ModelKit.Variable,1}, ::Array{Float64,1}, ::Array{Array{Int64,1},1}, ::Array{Any,1}) at C:\Users\matth\equilibriaTensegrityFramework.jl\equilibriaTensegrityFramework.jl\src\functionsForStableEquilibria.jl:31
 [26] start_demo(::Array{Float64,1}) at C:\Users\matth\equilibriaTensegrityFramework.jl\equilibriaTensegrityFramework.jl\src\functionsForStableEquilibria.jl:292
 [27] top-level scope at C:\Users\matth\equilibriaTensegrityFramework.jl\equilibriaTensegrityFramework.jl\src\tensegrityEquilibria.jl:7
 [28] include(::String) at .\client.jl:457
 [29] top-level scope at none:1
in expression starting at C:\Users\matth\equilibriaTensegrityFramework.jl\equilibriaTensegrityFramework.jl\src\tensegrityEquilibria.jl:7

This particular error is produced by

startParams=randn(ComplexF64, nparameters(P))
res=solve(P, target_parameters=startParams)

Sometimes, rerunning the code solves the issue (I guess because there is a random initialization involved). The corresponding polynomial system P is given by

System of length 35
 35 variables: p₁, p₂, p₃, p₄, p₅, p₆, delta₁, delta₂, delta₃, delta₄, delta₅, delta₆, lambda₁, lambda₂, lambda₃, lambda₄, lambda₅, lambda₆, lambda₇, v₁, v₂, v₃, v₄, v₅, v₆, v₇, v₈, v₉, v₁₀, v₁₁, v₁₂, v₁₃, v₁₄, v₁₅, v₁₆
 8 parameters: a₁₋₁, a₂₋₁, a₃₋₁, a₁₋₂, a₂₋₂, a₃₋₂, b₁, b₂

 2*p₄*lambda₅ + 2*(-0.866025403784439 + p₄)*lambda₆ + 2*(0.866025403784438 + p₄)*lambda₇ - 2*(p₁ - p₄)*lambda₁
 2*(-1 + p₅)*lambda₅ + 2*(0.5 + p₅)*lambda₆ + 2*(0.5 + p₅)*lambda₇ - 2*(p₂ - p₅)*lambda₁
 2*p₆*lambda₅ + 2*p₆*lambda₆ + 2*p₆*lambda₇ - 2*(p₃ - p₆)*lambda₁
 -2*lambda₂*delta₁ - (1 - delta₁)
 -2*lambda₃*delta₂ - (1 - delta₂)
 -2*lambda₄*delta₃ - (1 - delta₃)
 -2*lambda₅*delta₄ - (1 - delta₄)
 -2*lambda₆*delta₅ - (1 - delta₅)
 -2*lambda₇*delta₆ - (1 - delta₆)
 -4.0 + (p₁ - p₄)^2 + (p₂ - p₅)^2 + (p₃ - p₆)^2
 -delta₁^2 + p₁^2 + p₃^2 + (-1 + p₂)^2
 -delta₂^2 + p₃^2 + (-0.866025403784439 + p₁)^2 + (0.5 + p₂)^2
 -delta₃^2 + p₃^2 + (0.5 + p₂)^2 + (0.866025403784438 + p₁)^2
 -delta₄^2 + p₄^2 + p₆^2 + (-1 + p₅)^2
 -delta₅^2 + p₆^2 + (-0.866025403784439 + p₄)^2 + (0.5 + p₅)^2
 -delta₆^2 + p₆^2 + (0.5 + p₅)^2 + (0.866025403784438 + p₄)^2
 2*p₄*v₁₄ + v₁₅*(-1.73205080756888 + 2*p₄) + (1.73205080756888 + 2*p₄)*v₁₆ + (-2*p₁ + 2*p₄)*v₁₀ + (2*lambda₁ + 2*lambda₅ + 2*lambda₆ + 2*lambda₇)*v₁
 (-2 + 2*p₅)*v₁₄ + (1.0 + 2*p₅)*v₁₅ + (1.0 + 2*p₅)*v₁₆ + (-2*p₂ + 2*p₅)*v₁₀ + (2*lambda₁ + 2*lambda₅ + 2*lambda₆ + 2*lambda₇)*v₂
 2*p₆*v₁₄ + 2*p₆*v₁₅ + 2*p₆*v₁₆ + (-2*p₃ + 2*p₆)*v₁₀ + (2*lambda₁ + 2*lambda₅ + 2*lambda₆ + 2*lambda₇)*v₃
 -2*v₁₁*delta₁ + v₄*(1 - 2*lambda₂)
 -2*v₁₂*delta₂ + v₅*(1 - 2*lambda₃)
 -2*v₁₃*delta₃ + v₆*(1 - 2*lambda₄)
 -2*v₁₄*delta₄ + v₇*(1 - 2*lambda₅)
 -2*v₁₅*delta₅ + v₈*(1 - 2*lambda₆)
 -2*v₁₆*delta₆ + v₉*(1 - 2*lambda₇)
 -2*(p₁ - p₄)*v₁ - 2*(p₂ - p₅)*v₂ - 2*(p₃ - p₆)*v₃
 -2*v₄*delta₁
 -2*v₅*delta₂
 -2*v₆*delta₃
 2*v₁*p₄ + 2*v₂*(-1 + p₅) + 2*v₃*p₆ - 2*v₇*delta₄
 2*v₁*(-0.866025403784439 + p₄) + 2*v₂*(0.5 + p₅) + 2*v₃*p₆ - 2*v₈*delta₅
 2*v₁*(0.866025403784438 + p₄) + 2*v₂*(0.5 + p₅) + 2*v₃*p₆ - 2*v₉*delta₆
 -1 - 0.481160723746646*v₁ + 0.780597212802205*v₁₀ + 1.45722731207576*v₁₁ + 0.228247265817305*v₁₂ + 0.0839779731063026*v₁₃ - 1.82488516406006*v₁₄ - 0.224474693522123*v₁₅ - 2.15550885540766*v₁₆ + 1.79513147636094*v₂ + 0.484994402317654*v₃ - 0.0174600366022241*v₄ - 1.58052197849557*v₅ - 1.24679156009183*v₆ - 0.142158337321007*v₇ + 1.35072136471882*v₈ - 1.15859223454497*v₉
 b₁ + p₁*a₁₋₁ + p₂*a₂₋₁ + p₃*a₃₋₁
 b₂ + p₁*a₁₋₂ + p₂*a₂₋₂ + p₃*a₃₋₂

in this case. This system comes from an application in material science and is derived in a way comparable to the package Catastrophe.jl by Sascha. I should maybe mention that I can compute the system from Zeeman.jl (and of course systems entered via manual input), but larger systems fail for me.

I was wondering, if this is a known issue, whether there is a workaround or if this problem is maybe more on the user's side (me)? For starters, I was unable to identify the issue in your code.

Thanks a bunch for your help!

[saschatimme]

Are you on a 32-bit System?

You can check this by looking at

julia> Int
Int64

[matthiashimmelmann]

Thanks for your quick reply, Sascha. Unfortunately, yes, it appears I am on a x64 system.. I feel dumb for not checking it first though.

julia> Int
Int64

[matthiashimmelmann]

In the meantime, I tried to reproduce this with a "small" working example, inspired by the 3-prism with a fixed equilateral triangle on the bottom and variable vertices on the top. The bars and cables all have fixed properties. The error occurs around every second run. Only the z-coordinate of one vertex is chosen as control parameter.

### INIT
using HomotopyContinuation
@var p[1:9]
vertices=[[0,1,0],[sin(2*pi/3),cos(2*pi/3),0], [sin(4*pi/3),cos(4*pi/3),0], [p[1],p[2],p[3]], [p[4],p[5],p[6]], [p[7],p[8],p[9]]]
unknownBars=[[1,4,3], [2,5,3], [3,6,3]]
unknownCables=[[1,5,2.5,1.0], [2,6,2.5,1.0], [3,4,2.5,1.0], [4,5,1,1.0], [5,6,1,1.0], [4,6,1,1.0]]
listOfInternalVariables=p[1:8]

### CODE
@var delta[1:length(unknownCables)]
B, C, G = [], [], []
foreach(bar->push!(B, sum((vertices[Int64(bar[1])]-vertices[Int64(bar[2])]).^2)-bar[3]^2), unknownBars)
foreach(index->push!(C, sum((vertices[Int64(unknownCables[index][1])]-vertices[Int64(unknownCables[index][2])]).^2)-delta[index]^2), 1:length(unknownCables))
append!(G,B); append!(G,C);
e = map(t->t[4], unknownCables);
r = map(t->t[3], unknownCables);
Q = sum(e.*(r-delta).^2/2)
@var lambda[1:length(G)]
L = Q + lambda'*G
∇L = differentiate(L, [listOfInternalVariables; delta; lambda])
F=System(∇L; variables = [listOfInternalVariables; delta; lambda], parameters=[p[9]])
param = randn(ComplexF64, nparameters(F))
res=solve(F, target_parameters=param)

[saschatimme]

@matthiashimmelmann I think I pushed a fix on #460. Since I don't have a 32-bit system anymore, I cannot test this. Could you see whether this works for you? You can install the branch by doing

using Pkg
pkg"add HomotopyContinuation#32-bit

As a side note, if you are looking into larger problems, it could very well be that you have to use monodromy_solve instead of solve. For this would have to keep some more parameters around (at least to compute an initial set of solutions).

[matthiashimmelmann]

I don't think so?

(@v1.5) pkg> add HomotopyContinuation#32-bit
    Cloning git-repo `https://github.com/JuliaHomotopyContinuation/HomotopyContinuation.jl.git`
   Updating git-repo `https://github.com/JuliaHomotopyContinuation/HomotopyContinuation.jl.git`
  Resolving package versions...
Updating `C:\Users\matth\.julia\environments\v1.5\Project.toml`
  [f213a82b] ~ HomotopyContinuation v2.5.0 ⇒ v2.5.0 `https://github.com/JuliaHomotopyContinuation/HomotopyContinuation.jl.git#32-bit`
Updating `C:\Users\matth\.julia\environments\v1.5\Manifest.toml`
  [f213a82b] ~ HomotopyContinuation v2.5.0 ⇒ v2.5.0 `https://github.com/JuliaHomotopyContinuation/HomotopyContinuation.jl.git#32-bit`

julia> include("minimalWorkingEx.jl")
Computing mixed cells... 2024    Time: 0:00:04
  mixed_volume:  432128
ERROR: LoadError: InexactError: trunc(Int32, 149086040112)

I'll look into monodromy_solve though. Thanks for the advice!

[saschatimme]

mhm I don't have access to any 32-bit machine so this is tricky for me.
Just to make sure: You executed the code above in a new REPL session, did you?
And if so, can you show me stack trace again?

[matthiashimmelmann]

Yes, I did start a new Julia session and it recompiled HC after the update. The stacktrace remains the same (see below). It occured when I ran the code for the third time (first two went through seemlessly). System information states that my machine is 64-bit-based, which makes it hard for me to understand why this happens.. So you cannot reproduce the error on your machine?

julia> include("minimalWorkingEx.jl")
Computing mixed cells... 1392    Time: 0:00:03
  mixed_volume:  432128
ERROR: LoadError: InexactError: trunc(Int32, 21401280927447858)
Stacktrace:
 [1] throw_inexacterror(::Symbol, ::Type{Int32}, ::Int64) at .\boot.jl:558
 [2] checked_trunc_sint at .\boot.jl:580 [inlined]
 [3] toInt32 at .\boot.jl:617 [inlined]
 [4] Int32 at .\boot.jl:707 [inlined]
 [5] convert at .\number.jl:7 [inlined]
 [6] cconvert at .\essentials.jl:388 [inlined]
 [7] set_si! at .\gmp.jl:201 [inlined]
 [8] _broadcast_getindex_evalf at .\broadcast.jl:648 [inlined]
 [9] _broadcast_getindex at .\broadcast.jl:621 [inlined]
 [10] getindex at .\broadcast.jl:575 [inlined]
 [11] macro expansion at .\broadcast.jl:932 [inlined]
 [12] macro expansion at .\simdloop.jl:77 [inlined]
 [13] copyto! at .\broadcast.jl:931 [inlined]
 [14] copyto! at .\broadcast.jl:886 [inlined]
 [15] copy at .\broadcast.jl:862 [inlined]
 [16] materialize at .\broadcast.jl:837 [inlined]
 [17] solve!(::HomotopyContinuation.BinomialSystemSolver) at C:\Users\matth\.julia\packages\HomotopyContinuation\5xHVY\src\binomial_system.jl:179
 [18] solve at C:\Users\matth\.julia\packages\HomotopyContinuation\5xHVY\src\binomial_system.jl:163 [inlined]
 [19] iterate(::PolyhedralStartSolutionsIterator, ::Tuple{Int64,Int64}) at C:\Users\matth\.julia\packages\HomotopyContinuation\5xHVY\src\polyhedral.jl:81
 [20] _collect(::UnitRange{Int64}, ::PolyhedralStartSolutionsIterator, ::Base.HasEltype, ::Base.SizeUnknown) at .\array.jl:636
 [21] collect at .\array.jl:624 [inlined]
 [22] #solve#277 at C:\Users\matth\.julia\packages\HomotopyContinuation\5xHVY\src\solve.jl:501 [inlined]
 [23] solve(::System; show_progress::Bool, threading::Bool, catch_interrupt::Bool, target_parameters::Array{Complex{Float64},1}, stop_early_cb::Function, transform_result::Nothing, transform_parameters::typeof(identity), flatten::Nothing, target_subspaces::Nothing, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) 
at C:\Users\matth\.julia\packages\HomotopyContinuation\5xHVY\src\solve.jl:487
 [24] top-level scope at C:\Users\matth\minimalWorkingEx.jl:23
 [25] include(::String) at .\client.jl:457
 [26] top-level scope at none:1
in expression starting at C:\Users\matth\minimalWorkingEx.jl:23

[saschatimme]

I started to run CI tests on windows and there I can see that our tests are partially failing on windows with similar errors to yours. My suspicion is that GMP defines its integers only as longs which are on windows 32-bit but on linux and mac os 64-bit ...
So I probably have to look carefully to find the places where things go wrong

[saschatimme]

I just tagged v2.5.1 after merging #460. This should fix the problems you encountered :)

[matthiashimmelmann]

Yes, it appears you fixed it. Thank you very much!

[saschatimme]

You are welcome. Don't whether you already saw it, but this should be interesting for you: https://www.mis.mpg.de/calendar/conferences/2021/sanna2021.html

Hi,

I just wanted to report a similar issue in the latest version of the package. I updated my Julia to 1.10.4, as well as the HomotopyContinuation package to version 2.9.4 and get the error and stacktrace below. As previously mentioned in this thread, the Base.Clong type is indeed 32-bit on my 64-bit Windows machine. The issue arose when trying to run my colleague's code, who uses a Mac. He is able to run the same code without any problems, so there is definitely a difference in behaviour depending on the system used.

I might just be doing something wrong though. Is there a quick way to fix this perhaps?

Thank you in advance for your help!

ERROR: InexactError: trunc(Int32, -424523560724757)
Stacktrace:
  [1] throw_inexacterror(f::Symbol, ::Type{Int32}, val::Int64)
    @ Core .\boot.jl:634
  [2] checked_trunc_sint
    @ .\boot.jl:656 [inlined]
  [3] toInt32
    @ .\boot.jl:693 [inlined]
  [4] Int32
    @ .\boot.jl:783 [inlined]
  [5] convert
    @ .\number.jl:7 [inlined]
  [6] cconvert
    @ .\essentials.jl:543 [inlined]
  [7] set_si!
    @ .\gmp.jl:211 [inlined]
  [8] _broadcast_getindex_evalf
    @ .\broadcast.jl:709 [inlined]
  [9] _broadcast_getindex
    @ .\broadcast.jl:682 [inlined]
 [10] getindex
    @ .\broadcast.jl:636 [inlined]
 [11] macro expansion
    @ .\broadcast.jl:1004 [inlined]
 [12] macro expansion
    @ .\simdloop.jl:77 [inlined]
 [13] copyto!
    @ .\broadcast.jl:1003 [inlined]
 [14] copyto!
    @ .\broadcast.jl:956 [inlined]
 [15] copy
    @ .\broadcast.jl:928 [inlined]
 [16] materialize
    @ .\broadcast.jl:903 [inlined]
 [17] solve!(BSS::HomotopyContinuation.BinomialSystemSolver)
    @ HomotopyContinuation C:\Users\lvanpouc\.julia\packages\HomotopyContinuation\HHk7W\src\binomial_system.jl:178
 [18] solve
    @ C:\Users\lvanpouc\.julia\packages\HomotopyContinuation\HHk7W\src\binomial_system.jl:162 [inlined]
 [19] iterate(iter::PolyhedralStartSolutionsIterator, ::Tuple{Int64, Int64})
    @ HomotopyContinuation C:\Users\lvanpouc\.julia\packages\HomotopyContinuation\HHk7W\src\polyhedral.jl:80
 [20] _collect(cont::UnitRange{Int64}, itr::PolyhedralStartSolutionsIterator, ::Base.HasEltype, isz::Base.SizeUnknown)
    @ Base .\array.jl:772
 [21] collect
    @ .\array.jl:759 [inlined]
    @ C:\Users\lvanpouc\.julia\packages\HomotopyContinuation\HHk7W\src\solve.jl:501 [inlined]
 [23] solve(args::System; show_progress::Bool, threading::Bool, catch_interrupt::Bool, target_parameters::Nothing, stop_early_cb::Function, transform_result::Nothing, transform_parameters::typeof(identity), flatten::Nothing, target_subspaces::Nothing, kwargs::@Kwargs{…})
    @ HomotopyContinuation C:\Users\lvanpouc\.julia\packages\HomotopyContinuation\HHk7W\src\solve.jl:487
 [24] macro expansion
    @ .\timing.jl:361 [inlined]
 [25] top-level scope
    @ c:\Users\lvanpouc\Documents\julia\System identification\walsh.jl:52
Some type information was truncated. Use `show(err)` to see complete types.

[saschatimme]

If your coefficients are small than 2^53 you can convert without loss the coefficient to floating point (double precision)