Fail to install SoS in Windows

[HenryLeongStat]

Run the following command in powershell:
PS C:\Users\ml69> pip install sos sos-notebook

Got error (the bottom):

Collecting sos
  Using cached sos-0.9.12.8-py3-none-any.whl
Collecting sos-notebook
  Using cached sos_notebook-0.9.12.9-py3-none-any.whl
Requirement already satisfied: pygments in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Requirement already satisfied: tqdm in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Requirement already satisfied: pexpect in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Requirement already satisfied: psutil in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Collecting xxhash (from sos)
  Using cached xxhash-1.0.1.zip
Collecting pydot (from sos)
Requirement already satisfied: pyyaml in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Requirement already satisfied: fasteners in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Requirement already satisfied: colorama; sys_platform == "win32" in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Requirement already satisfied: networkx in c:\users\ml69\anaconda3\lib\site-packages (from sos)
Collecting pydotplus (from sos)
Collecting markdown (from sos-notebook)
  Using cached Markdown-2.6.11-py2.py3-none-any.whl
Requirement already satisfied: notebook>=5.0.0 in c:\users\ml69\anaconda3\lib\site-packages (from sos-notebook)
Collecting tabulate (from sos-notebook)
Collecting wand (from sos-notebook)
Requirement already satisfied: ipython in c:\users\ml69\anaconda3\lib\site-packages (from sos-notebook)
Requirement already satisfied: nbconvert>=5.1.1 in c:\users\ml69\anaconda3\lib\site-packages (from sos-notebook)
Requirement already satisfied: ipykernel in c:\users\ml69\anaconda3\lib\site-packages (from sos-notebook)
Requirement already satisfied: nbformat in c:\users\ml69\anaconda3\lib\site-packages (from sos-notebook)
Requirement already satisfied: ptyprocess>=0.5 in c:\users\ml69\anaconda3\lib\site-packages (from pexpect->sos)
Requirement already satisfied: pyparsing>=2.1.4 in c:\users\ml69\anaconda3\lib\site-packages (from pydot->sos)
Requirement already satisfied: monotonic>=0.1 in c:\users\ml69\anaconda3\lib\site-packages (from fasteners->sos)
Requirement already satisfied: six in c:\users\ml69\anaconda3\lib\site-packages (from fasteners->sos)
Requirement already satisfied: decorator>=4.1.0 in c:\users\ml69\anaconda3\lib\site-packages (from networkx->sos)
Requirement already satisfied: setuptools>=18.5 in c:\users\ml69\anaconda3\lib\site-packages (from ipython->sos-notebook)
Requirement already satisfied: jedi>=0.10 in c:\users\ml69\anaconda3\lib\site-packages (from ipython->sos-notebook)
Requirement already satisfied: pickleshare in c:\users\ml69\anaconda3\lib\site-packages (from ipython->sos-notebook)
Requirement already satisfied: simplegeneric>0.8 in c:\users\ml69\anaconda3\lib\site-packages (from ipython->sos-notebook)
Requirement already satisfied: traitlets>=4.2 in c:\users\ml69\anaconda3\lib\site-packages (from ipython->sos-notebook)
Requirement already satisfied: prompt_toolkit<2.0.0,>=1.0.4 in c:\users\ml69\anaconda3\lib\site-packages (from ipython->sos-notebook)
Requirement already satisfied: mistune>=0.7.4 in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: jinja2 in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: jupyter_core in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: entrypoints>=0.2.2 in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: bleach in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: pandocfilters>=1.4.1 in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: testpath in c:\users\ml69\anaconda3\lib\site-packages (from nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: jupyter_client in c:\users\ml69\anaconda3\lib\site-packages (from ipykernel->sos-notebook)
Requirement already satisfied: tornado>=4.0 in c:\users\ml69\anaconda3\lib\site-packages (from ipykernel->sos-notebook)
Requirement already satisfied: ipython_genutils in c:\users\ml69\anaconda3\lib\site-packages (from nbformat->sos-notebook)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in c:\users\ml69\anaconda3\lib\site-packages (from nbformat->sos-notebook)
Requirement already satisfied: wcwidth in c:\users\ml69\anaconda3\lib\site-packages (from prompt_toolkit<2.0.0,>=1.0.4->ipython->sos-notebook)
Requirement already satisfied: MarkupSafe>=0.23 in c:\users\ml69\anaconda3\lib\site-packages (from jinja2->nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: html5lib>=0.99999999 in c:\users\ml69\anaconda3\lib\site-packages (from bleach->nbconvert>=5.1.1->sos-notebook)
Requirement already satisfied: pyzmq>=13 in c:\users\ml69\anaconda3\lib\site-packages (from jupyter_client->ipykernel->sos-notebook)
Requirement already satisfied: python-dateutil>=2.1 in c:\users\ml69\anaconda3\lib\site-packages (from jupyter_client->ipykernel->sos-notebook)
Requirement already satisfied: webencodings in c:\users\ml69\anaconda3\lib\site-packages (from html5lib>=0.99999999->bleach->nbconvert>=5.1.1->sos-notebook)
Building wheels for collected packages: xxhash
  Running setup.py bdist_wheel for xxhash ... error
  Complete output from command C:\Users\ml69\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\ml69\\AppData\\Local\\Temp\\pip-build-wlz8q2on\\xxhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\ml69\AppData\Local\Temp\tmpnaujf4dcpip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.6
  creating build\lib.win-amd64-3.6\xxhash
  copying xxhash\cffi.py -> build\lib.win-amd64-3.6\xxhash
  copying xxhash\__init__.py -> build\lib.win-amd64-3.6\xxhash
  running build_ext
  building 'cpython' extension
  error: [WinError 3] The system cannot find the path specified: 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\PlatformSDK\\lib'

  ----------------------------------------
  Failed building wheel for xxhash
  Running setup.py clean for xxhash
Failed to build xxhash
Installing collected packages: xxhash, pydot, pydotplus, sos, markdown, tabulate, wand, sos-notebook
  Running setup.py install for xxhash ... error
    Complete output from command C:\Users\ml69\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\ml69\\AppData\\Local\\Temp\\pip-build-wlz8q2on\\xxhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\ml69\AppData\Local\Temp\pip-qczbq2i7-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\xxhash
    copying xxhash\cffi.py -> build\lib.win-amd64-3.6\xxhash
    copying xxhash\__init__.py -> build\lib.win-amd64-3.6\xxhash
    running build_ext
    building 'cpython' extension
    **error: [WinError 3] The system cannot find the path specified: 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\PlatformSDK\\lib'

    ----------------------------------------
Command "C:\Users\ml69\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\ml69\\AppData\\Local\\Temp\\pip-build-wlz8q2on\\xxhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\ml69\AppData\Local\Temp\pip-qczbq2i7-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\ml69\AppData\Local\Temp\pip-build-wlz8q2on\xxhash\**

Information of this machine:

[HenryLeongStat]

Also, should I open another ticket for changing the installation guide for windows user?

[BoPeng]

This is related to #875 in which we replaced the system hash module with a third-party one. This module needs to be compiled from source, which is usually not a problem but is causing problem under windows.

My windows machine has VC installed and can install xxhash, therefore sos without problem.

@gaow any comment?

[HenryLeongStat]

I also installed VC on this machine. Let me try to install xxhash

[HenryLeongStat]

Fail to do so...

PS C:\Users\ml69> pip install xxhash
Collecting xxhash
  Using cached xxhash-1.0.1.zip
Building wheels for collected packages: xxhash
  Running setup.py bdist_wheel for xxhash ... error
  Complete output from command C:\Users\ml69\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\ml69\\AppData\\Local\\Temp\\pip-build-t3ea_pri\\xxhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\ml69\AppData\Local\Temp\tmpwrnvk6zjpip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.6
  creating build\lib.win-amd64-3.6\xxhash
  copying xxhash\cffi.py -> build\lib.win-amd64-3.6\xxhash
  copying xxhash\__init__.py -> build\lib.win-amd64-3.6\xxhash
  running build_ext
  building 'cpython' extension
  error: [WinError 3] The system cannot find the path specified: 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\PlatformSDK\\lib'

  ----------------------------------------
  Failed building wheel for xxhash
  Running setup.py clean for xxhash
Failed to build xxhash
Installing collected packages: xxhash
  Running setup.py install for xxhash ... error
    Complete output from command C:\Users\ml69\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\ml69\\AppData\\Local\\Temp\\pip-build-t3ea_pri\\xxhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\ml69\AppData\Local\Temp\pip-45daqtxo-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\xxhash
    copying xxhash\cffi.py -> build\lib.win-amd64-3.6\xxhash
    copying xxhash\__init__.py -> build\lib.win-amd64-3.6\xxhash
    running build_ext
    building 'cpython' extension
    error: [WinError 3] The system cannot find the path specified: 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\PlatformSDK\\lib'

    ----------------------------------------
Command "C:\Users\ml69\anaconda3\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\ml69\\AppData\\Local\\Temp\\pip-build-t3ea_pri\\xxhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\ml69\AppData\Local\Temp\pip-45daqtxo-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\ml69\AppData\Local\Temp\pip-build-t3ea_pri\xxhash\

[BoPeng]

They need specific version of VC that is compatible with Python. From the error message, it is looking for The system cannot find the path specified: 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\PlatformSDK\\lib. Do you have VS 14.0 installed?

[HenryLeongStat]

Yes, I have it installed. Just google it and people said that \\PlatformSDK\\lib won't exist unless having SDK installed. Waiting for the installation of SDK... It takes couple hours.

[BoPeng]

So we can either switch back to hash on windows, or require users to install VC. I personally would prefer the former.

[HenryLeongStat]

If requiring users to install VC and SDK, it might take a lot of disk space and time, and some of the users might never use VC and SDK (it should be the case in my office...)

So we can either switch back to hash on windows,

Do you mean keeping xxhash for LINUX and macOS users, or also using hash for them?

[BoPeng]

That is my plan but I will wait for @gaow 's comment. Basically we will have to conditionally use xxhash in setup.py, and modify 700ea2d

[gaow]

Sorry about the unexpected troubles with xxhash. I guess it is a judgement call whether xxhash's benefit in speeding up hashing is worth treating separately, because that will make cross windows / non-windows signatures not portable. But it is not an issue for now since we are still pending on #850 ? Since windows is much less likely used in large scale analysis that are expected to be portable, I'd support switching back to hash on Windows.

[BoPeng]

Also pending on #899 since right now signatures are verified locally, not across network to a centralized server. Given the status of sos on windows, I think it makes sense to make it easier to install.