_pydstool_jl_hack - where is this?

[jacobsca]

I'm trying to use PyDSTool in DifferentialEquations...but it won't load. I get this long red error message that says

ERROR: InitError: PyError (PyImport_ImportModule

The Python package _pydstool_jl_hack could not be found by pyimport. Usually this means
that you did not install _pydstool_jl_hack in the Python version being used by PyCall.

Any clues how to fix this?? I done some web crawling but cannot seemingly find a helpful answer...

[ChrisRackauckas]

I'm not sure how to fix this. It's pointing to the monkey patching here: https://github.com/JuliaDiffEq/PyDSTool.jl/blob/master/src/_pydstool_jl_hack.py . @tkf was the fix tagged?

[tkf]

@ChrisRackauckas No release AFAICT:
https://github.com/robclewley/pydstool/releases
https://pypi.org/project/PyDSTool/#history
https://anaconda.org/conda-forge/pydstool

Actually this "should not" happen. But I realized that it can happen if pydstool or scipy is not installed. I wasn't handling such case properly in __init__ of PyDSTool.jl.

@jacobsca Maybe you are not using Conda.jl? What is the output of the following commands?:

versioninfo()
using Pkg
Pkg.status()
using PyCall  # if it errors, try Pkg.add("PyCall") first
@show PyCall.conda
@show PyCall.pyprogramname

If PyCall.conda is false or PyCall.pyprogramname points to Python 3, you can try

using Pkg
ENV["CONDA_JL_VERSION"] = "2"
ENV["PYTHON"] = ""
Pkg.build("PyCall")  # let PyCall.jl use Conda.jl
exit()

to let Conda.jl download miniconda2 and then install pydstool in it.

[jacobsca]

Sorry - most of this is in the "stupid category." I've had PyDSTool up and running  - but under 2.7. I have moved most of my work to 3.6 but forgot to do this with PyDSTool. I installed it - the examples in Julia now work. :) Its not for DifferentialEquations and not really about the functionality but there is a nit - and nits bother me. I cannot get Radau or Dopri to run - I cannot get the compilation to work. I get the following error - maybe you have seen this?? ------------------ clasajacobson@clasajacobson-Oryx-Pro:~/Downloads/pydstool-master/examples$ python3 Dopri_backwards_test.py

…

-----------------------------------------------------------Present limitation of Python: Cannot rebuild librarywithout exiting Python and deleting the shared library   /home/clasajacobson/Downloads/pydstool-master/examples/_dop853_my_ode_vf.soby hand! If you made any changes to the system you shouldnot proceed with running the integrator until you quitand rebuild.----------------------------------------------------------- Did not compile shared library.Error in importing compiled vector field and integrator.Did you compile the RHS C code?Traceback (most recent call last):  File "dop853_temp/dop853_my_ode_vf.py", line 14, in swig_import_helper    return importlib.import_module(mname)  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module    return _bootstrap._gcd_import(name[level:], package, level)  File "<frozen importlib._bootstrap>", line 994, in _gcd_import  File "<frozen importlib._bootstrap>", line 971, in _find_and_load  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlockedModuleNotFoundError: No module named '_dop853_my_ode_vf' During handling of the above exception, another exception occurred: Traceback (most recent call last):  File "Dopri_backwards_test.py", line 23, in <module>    trajf = ode.compute('testf')  File "/usr/local/lib/python3.6/dist-packages/PyDSTool-0.90.2-py3.6.egg/PyDSTool/Generator/Dopri_ODEsystem.py", line 463, in compute    extraSpace=self.algparams['extraspace'],  File "/usr/local/lib/python3.6/dist-packages/PyDSTool-0.90.2-py3.6.egg/PyDSTool/Generator/Dopri_ODEsystem.py", line 45, in __init__    modname, *imp.find_module(modname, ["dop853_temp"]))  File "/usr/lib/python3.6/imp.py", line 235, in load_module    return load_source(name, filename, file)  File "/usr/lib/python3.6/imp.py", line 172, in load_source    module = _load(spec)  File "<frozen importlib._bootstrap>", line 684, in _load  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked  File "<frozen importlib._bootstrap_external>", line 678, in exec_module  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed  File "dop853_temp/dop853_my_ode_vf.py", line 17, in <module>    _dop853_my_ode_vf = swig_import_helper()  File "dop853_temp/dop853_my_ode_vf.py", line 16, in swig_import_helper    return importlib.import_module('_dop853_my_ode_vf')  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module    return _bootstrap._gcd_import(name[level:], package, level)  File "<frozen importlib._bootstrap>", line 994, in _gcd_import  File "<frozen importlib._bootstrap>", line 971, in _find_and_load  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlockedModuleNotFoundError: No module named '_dop853_my_ode_vf'clasajacobson@clasajacobson-Oryx-Pro:~/Downloads/pydstool-master/examples$  Clas A. Jacobson1010 Horizon WayCT 06084 860 783 0074 (home) jacobsonclas@sbcglobal.net On Thursday, November 1, 2018, 1:22:08 AM EDT, Christopher Rackauckas <notifications@github.com> wrote: I'm not sure how to fix this. It's pointing to the monkey patching here: https://github.com/JuliaDiffEq/PyDSTool.jl/blob/master/src/_pydstool_jl_hack.py . @tkf was the fix tagged? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ChrisRackauckas]

Radau and Dopri needs a special install for PyDSTool that we don't do by default. These need to be compiled separately and something.

[HenriLaurie]

I think this is still unresolved. In my course at the moment we can't use PyDSTool because of both Python 2.x and Python 3.x being installed on the system. PyCall seems to be defaulting to Python 3.x and then using PyDSTool fails.

And often but not always it asks for _pdstool_jl_hack to be installed.

It isn't clear to me what we should do ... I am reluctant to play with ENV() because I worry it will contaminate other uses of PyCall.

[tkf]

I think using Python 2 is the only option unless you want to use pydstool master. You need to use ENV for it since it is the official way to configure Python used for PyCall https://github.com/JuliaPy/PyCall.jl#specifying-the-python-version (i.e., ENV["PYTHON"] = "python2"; using Pkg; Pkg.build("PyCall")). As you worried, this will "contaminate" the PyCall setup because there is no way to use Python 2 without switching to it entirely.

[LolianSh]

Hi, I am facing a similar problem.

I can add and build the PyDSTool.jl but when I test it this what I get:

[ Info: Installing PyDSTool via the Conda pydstool package...
[ Info: Running `conda config --add channels conda-forge --file /Users/loliansh/.julia/conda/3/condarc-julia.yml --force` in root environment
Warning: 'conda-forge' already in 'channels' list, moving to the top
[ Info: Running `conda install -y pydstool` in root environment
Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

┌ Warning: `getindex(o::PyObject, s::AbstractString)` is deprecated in favor of dot overloading (`getproperty`) so elements should now be accessed as e.g. `o."s"` instead of `o["s"]`.
│   caller = __init__() at PyDSTool.jl:23
└ @ PyDSTool ~/.julia/packages/PyDSTool/T2R3A/src/PyDSTool.jl:23
ERROR: InitError: PyError (PyImport_ImportModule

The Python package _pydstool_jl_hack could not be found by pyimport. Usually this means
that you did not install _pydstool_jl_hack in the Python version being used by PyCall.

PyCall is currently configured to use the Julia-specific Python distribution
installed by the Conda.jl package.  To install the _pydstool_jl_hack module, you can
use `pyimport_conda("_pydstool_jl_hack", PKG)`, where PKG is the Anaconda
package the contains the module _pydstool_jl_hack, or alternatively you can use the
Conda package directly (via `using Conda` followed by `Conda.add` etcetera).

Alternatively, if you want to use a different Python distribution on your
system, such as a system-wide Python (as opposed to the Julia-specific Python),
you can re-configure PyCall with that Python.   As explained in the PyCall
documentation, set ENV["PYTHON"] to the path/name of the python executable
you want to use, run Pkg.build("PyCall"), and re-launch Julia.

) <class 'ImportError'>
ImportError("cannot import name 'factorial' from 'scipy' (/Users/loliansh/.julia/conda/3/lib/python3.7/site-packages/scipy/__init__.py)")
  File "/Users/loliansh/.julia/packages/PyDSTool/T2R3A/src/_pydstool_jl_hack.py", line 29, in <module>
    import PyDSTool
  File "/Users/loliansh/.julia/conda/3/lib/python3.7/site-packages/PyDSTool/__init__.py", line 60, in <module>
    from .Events import *
  File "/Users/loliansh/.julia/conda/3/lib/python3.7/site-packages/PyDSTool/Events.py", line 13, in <module>
    from .Variable import *
  File "/Users/loliansh/.julia/conda/3/lib/python3.7/site-packages/PyDSTool/Variable.py", line 10, in <module>
    from .utils import *
  File "/Users/loliansh/.julia/conda/3/lib/python3.7/site-packages/PyDSTool/utils.py", line 10, in <module>
    from .common import *
  File "/Users/loliansh/.julia/conda/3/lib/python3.7/site-packages/PyDSTool/common.py", line 45, in <module>
    from scipy import factorial

Stacktrace:
 [1] pyimport(::String) at /Users/loliansh/.julia/packages/PyCall/ttONZ/src/PyCall.jl:544
 [2] __init__() at /Users/loliansh/.julia/packages/PyDSTool/T2R3A/src/PyDSTool.jl:24
 [3] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:692
 [4] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:776
 [5] _require(::Base.PkgId) at ./loading.jl:1001
 [6] require(::Base.PkgId) at ./loading.jl:922
 [7] require(::Module, ::Symbol) at ./loading.jl:917

PyCall points to Python3 in my case too, on OSX.

Do you have any suggestions on how to get it working ?

[ChrisRackauckas]

I think that we should no longer recommend PyDSTool in the bifurcation section of the docs. If @tkf and @rveltz want to write examples of using the pure Julia bifurcation tools with ODEProblems, I'd prefer to do that since it seems pydstool on the Python side won't be updating any time soon.

[rveltz]

Thank you for your suggestion! I still have an issue on my side concerning the wrapper. I wrote a message on slack about it. Once, this is fixed, it is very easy for me to write a tutorial.

As for Bifurcations.jl, it still lacks the computation of Floquet compared to PyDSTool if I am not mistaken. In this case (hope I am wrong), I am not sure we sure whether we should not mention PyDSTool anymore.

[tkf]

As for Bifurcations.jl, it still lacks the computation of Floquet compared to PyDSTool if I am not mistaken.

That's correct (sadly...). Also, IIRC, PyDSTool (or more like AUTO) calculates the Jacobian for the bifurcation analysis/continuation of the limit cycle in a clever way (and uses its intermediate result for Floquet or something like that). What I implemented there was super dumb (just use ForwardDiff).

[rveltz]

Yes they use a periodic schur decomposition.

Yesterday, I was trying Bifurcations.jl, there is so much in the test that is not mentioned in the docs, it is a pity. A proper README.md would bring more traffic as well. Considering the amount of work and quality of this package, you should consider adding basic things like Hopf_to_LC... in the docs.

What I implemented there was super dumb (just use ForwardDiff).

Do you mean that you have a way to get the Floquet coeffs? Even if not very accurate, you should add a quick and dirty solution, it can always be improved in the future.

[tkf]

there is so much in the test that is not mentioned in the docs

Yeah, that's true... The reason why I've been shy about this project was that I have hard time understanding some outputs. Maybe there are critical bugs somewhere or I just miss-set some algorithm parameters or my intuition is wrong... (In particular, tkf/Bifurcations.jl#8 was the one behaving strangely. Though I need to clean it up and explain the oddity before asking for a help.)

Maybe I should just document everything and let people find the bugs for me 😛

Do you mean that you have a way to get the Floquet coeffs?

I meant the computation of Jacobian is slow because I just simply hand it over to ForwardDiff.