`fpga_hls_hw_emu` fails in `vpl` step

[j-stephan]

I'm on the current sycl/unified/next branch and using Vitis 2020.2 with a xilinx_u200_xdma_201830_2 target. Compiling the single_task_vector_add test case works fine but it fails once vpl is called:

$ clang++ --gcc-toolchain=/trinity/shared/pkg/compiler/gcc/11.2.0 -std=c++20 -fsycl -fsycl-targets=fpga64_hls_hw_emu single_task_vector_add.cpp

[Snipped - Previous messages in attached log file]

****** vpl v2020.2 (64-bit)
  **** SW Build (by xbuild) on 2020-11-18-05:13:29
    ** Copyright 1986-2020 Xilinx, Inc. All Rights Reserved.

INFO: [VPL 60-839] Read in kernel information from file '/tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/kernel_info.dat'.
INFO: [VPL 74-74] Compiler Version string: 2020.2
INFO: [VPL 60-423]   Target device: xilinx_u200_xdma_201830_2
INFO: [VPL 60-1032] Extracting hardware platform to /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/vivado/vpl/.local/hw_platform
WARNING: /trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/tps/lnx64/jre9.0.4 does not exist.
[11:04:50] Run vpl: Step create_project: Started
Creating Vivado project.
[11:04:53] Run vpl: Step create_project: Completed
[11:04:53] Run vpl: Step create_bd: Started
[11:05:37] Run vpl: Step create_bd: Completed
[11:05:37] Run vpl: Step update_bd: Started
[11:05:38] Run vpl: Step update_bd: Completed
[11:05:38] Run vpl: Step generate_target: Started
[11:05:55] Run vpl: Step generate_target: Completed
[11:05:55] Run vpl: Step config_hw_emu.gen_scripts: Started
[11:06:14] Run vpl: Step config_hw_emu.gen_scripts: Completed
[11:06:14] Run vpl: Step config_hw_emu.compile: Started
[11:06:18] Run vpl: Step config_hw_emu.compile: Failed
[11:06:22] Run vpl: FINISHED. Run Status: config_hw_emu.compile ERROR

===>The following messages were generated while processing /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/vivado/vpl/prj/prj.sim/sim_1/behav_waveform/xsim :
ERROR: [VPL 43-3410] Failed to compile one of the generated C files.
Please recompile with "-mt off -v 1" switch to identify which design unit failed
.
ERROR: [VPL 60-773] In '/tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/vivado/vpl/vivado.log', caught Tcl error:  xsc -c --gcc_compile_options -DBOOST_SYSTEM_NO_DEPRECATED --gcc_compile_options -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/simmodels/xsim/2020.2/lnx64/6.2.0/ext/protobuf/include --gcc_compile_options -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_embedded_scheduler_sw_0_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_embedded_scheduler_sw_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/src/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/src/cpp_src -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/src/sysc_src -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_xtlm_simple_intercon_0_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_xtlm_simple_intercon_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_1/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_1/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_2/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_2/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_3/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_3/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_0_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_1_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_1_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_2_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_2_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_3_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_3_0/sim -I../../../../prj.ip_user_files/bd/emu/sim -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/tps/boost_1_64_0 -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/simmodels/xsim/2020.2/lnx64/6.2.0/ext/protobuf/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/xtlm/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/xtlm_simple_interconnect_v1_0/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/common_cpp_v1_0/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/emu_perf_common_v1_0/include -work xil_defaultlib -f emu_wrapper_xsc.prj
WARNING: [VPL 60-732] Link warning: No monitor points found for BD automation.
ERROR: [VPL 60-1328] Vpl run 'vpl' failed
ERROR: [VPL 60-806] Failed to finish platform linker
INFO: [v++ 60-1442] [11:06:22] Run run_link: Step vpl: Failed
Time (s): cpu = 00:01:32 ; elapsed = 00:01:56 . Memory (MB): peak = 1670.371 ; gain = 0.000 ; free physical = 338235 ; free virtual = 358341
ERROR: [v++ 60-661] v++ link run 'run_link' failed
ERROR: [v++ 60-626] Kernel link failed to complete
ERROR: [v++ 60-703] Failed to finish linking
INFO: [v++ 60-1653] Closing dispatch client.
Vitis linkage stage failed
Output /tmp/single_task_vector_add-47f9b65zz08txn/single_task_vector_add-47f9b6.xclbin was not properly produced by previous commands
clang-14: error: sycl-link-vxx command failed with exit code 255 (use -v to see invocation)
clang version 14.0.0 (https://github.com/triSYCL/sycl.git 0d2c6d5fca5e7822c8955955edc31e1acf74d4c0)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/stepha27/sycl-workspace/sycl-build/bin
clang-14: note: diagnostic msg: Error generating preprocessed source(s).

Any ideas on how to proceed from here? Can this error be debugged further by passing additional flags or something?

Log file starts here

$ clang++ --gcc-toolchain=/trinity/shared/pkg/compiler/gcc/11.2.0 -std=c++20 -fsycl -fsycl-targets=fpga64_hls_hw_emu single_task_vector_add.cpp
Found Vitis version 2020.2
warning: Linking two modules of different target triples: '/trinity/shared/pkg/devel/sdaccel/Vitis_HLS/2020.2/lnx64/lib/libspir64-39-hls.bc' is 'fpga64-xilinx-none' whereas 'llvm-link' is 'fpga64_hls_hw_emu-xilinx-linux'

Option Map File Used: '/trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/data/vitis/vpp/optMap.xml'

****** v++ v2020.2 (64-bit)
  **** SW Build (by xbuild) on 2020-11-18-05:13:29
    ** Copyright 1986-2020 Xilinx, Inc. All Rights Reserved.

INFO: [v++ 60-1306] Additional information associated with this v++ compile can be found at:
	Reports: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_comp_report/handlerEE_clES2_E3add_eCM5g51B
	Log files: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_comp_log/handlerEE_clES2_E3add_eCM5g51B
Running Dispatch Server on port:40473
INFO: [v++ 60-1548] Creating build summary session with primary output /tmp/single_task_vector_add-47f9b65zz08txn/handlerEE_clES2_E3add_eCM5g51B.xo.compile_summary, at Wed Mar  2 11:03:01 2022
INFO: [v++ 60-1316] Initiating connection to rulecheck server, at Wed Mar  2 11:03:01 2022
Running Rule Check Server on port:40259
INFO: [v++ 60-1315] Creating rulecheck session with output '/tmp/single_task_vector_add-47f9b65zz08txn/vxx_comp_report/handlerEE_clES2_E3add_eCM5g51B/v++_compile_handlerEE_clES2_E3add_eCM5g51B_guidance.html', at Wed Mar  2 11:03:03 2022
INFO: [v++ 60-895]   Target platform: /trinity/shared/pkg/devel/sdaccel/platforms/xilinx_u200_xdma_201830_2/xilinx_u200_xdma_201830_2.xpfm
INFO: [v++ 60-1578]   This platform contains Device Support Archive '/trinity/shared/pkg/devel/sdaccel/platforms/xilinx_u200_xdma_201830_2/hw/xilinx_u200_xdma_201830_2.dsa'
INFO: [v++ 74-74] Compiler Version string: 2020.2
INFO: [v++ 60-1302] Platform 'xilinx_u200_xdma_201830_2.xpfm' has been explicitly enabled for this release.
INFO: [v++ 60-585] Compiling for hardware emulation target
INFO: [v++ 60-423]   Target device: xilinx_u200_xdma_201830_2
INFO: [v++ 60-242] Creating kernel: 'handlerEE_clES2_E3add_eCM5g51B'
WARNING: /trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/tps/lnx64/jre9.0.4 does not exist.

===>The following messages were generated while  performing high-level synthesis for kernel: handlerEE_clES2_E3add_eCM5g51B Log file: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_comp_tmp/handlerEE_clES2_E3add_eCM5g51B/handlerEE_clES2_E3add_eCM5g51B/vitis_hls.log :
INFO: [v++ 204-61] Pipelining loop 'Loop 1'.
INFO: [v++ 200-1470] Pipelining result : Target II = 1, Final II = 1, Depth = 3, loop 'Loop 1'
INFO: [v++ 204-61] Pipelining loop 'Loop 2'.
INFO: [v++ 200-1470] Pipelining result : Target II = 1, Final II = 1, Depth = 3, loop 'Loop 2'
INFO: [v++ 200-790] **** Loop Constraint Status: All loop constraints were satisfied.
INFO: [v++ 200-789] **** Estimated Fmax: 411.00 MHz
INFO: [v++ 60-594] Finished kernel compilation
INFO: [v++ 60-244] Generating system estimate report...
INFO: [v++ 60-1092] Generated system estimate report: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_comp_report/handlerEE_clES2_E3add_eCM5g51B/system_estimate_handlerEE_clES2_E3add_eCM5g51B.xtxt
INFO: [v++ 60-586] Created /tmp/single_task_vector_add-47f9b65zz08txn/handlerEE_clES2_E3add_eCM5g51B.xo
INFO: [v++ 60-2343] Use the vitis_analyzer tool to visualize and navigate the relevant reports. Run the following command. 
    vitis_analyzer /tmp/single_task_vector_add-47f9b65zz08txn/handlerEE_clES2_E3add_eCM5g51B.xo.compile_summary 
INFO: [v++ 60-791] Total elapsed time: 0h 0m 54s
INFO: [v++ 60-1653] Closing dispatch client.
Option Map File Used: '/trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/data/vitis/vpp/optMap.xml'

****** v++ v2020.2 (64-bit)
  **** SW Build (by xbuild) on 2020-11-18-05:13:29
    ** Copyright 1986-2020 Xilinx, Inc. All Rights Reserved.

INFO: [v++ 60-1306] Additional information associated with this v++ link can be found at:
	Reports: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_report/link
	Log files: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_log/link
Running Dispatch Server on port:36559
INFO: [v++ 60-1548] Creating build summary session with primary output /tmp/single_task_vector_add-47f9b65zz08txn/single_task_vector_add-47f9b6.xclbin.link_summary, at Wed Mar  2 11:03:57 2022
INFO: [v++ 60-1316] Initiating connection to rulecheck server, at Wed Mar  2 11:03:57 2022
Running Rule Check Server on port:37952
INFO: [v++ 60-1315] Creating rulecheck session with output '/tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_report/link/v++_link_single_task_vector_add-47f9b6_guidance.html', at Wed Mar  2 11:03:59 2022
INFO: [v++ 60-895]   Target platform: /trinity/shared/pkg/devel/sdaccel/platforms/xilinx_u200_xdma_201830_2/xilinx_u200_xdma_201830_2.xpfm
INFO: [v++ 60-1578]   This platform contains Device Support Archive '/trinity/shared/pkg/devel/sdaccel/platforms/xilinx_u200_xdma_201830_2/hw/xilinx_u200_xdma_201830_2.dsa'
INFO: [v++ 74-74] Compiler Version string: 2020.2
INFO: [v++ 60-1302] Platform 'xilinx_u200_xdma_201830_2.xpfm' has been explicitly enabled for this release.
INFO: [v++ 60-629] Linking for hardware emulation target
INFO: [v++ 60-423]   Target device: xilinx_u200_xdma_201830_2
INFO: [v++ 60-1332] Run 'run_link' status: Not started
INFO: [v++ 60-1443] [11:04:07] Run run_link: Step system_link: Started
INFO: [v++ 60-1453] Command Line: system_link --xo /tmp/single_task_vector_add-47f9b65zz08txn/handlerEE_clES2_E3add_eCM5g51B.xo -keep --config /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/syslinkConfig.ini --xpfm /trinity/shared/pkg/devel/sdaccel/platforms/xilinx_u200_xdma_201830_2/xilinx_u200_xdma_201830_2.xpfm --target emu --output_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int --temp_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link
INFO: [v++ 60-1454] Run Directory: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/run_link
INFO: [SYSTEM_LINK 60-1316] Initiating connection to rulecheck server, at Wed Mar  2 11:04:09 2022
INFO: [SYSTEM_LINK 82-70] Extracting xo v3 file /tmp/single_task_vector_add-47f9b65zz08txn/handlerEE_clES2_E3add_eCM5g51B.xo
INFO: [SYSTEM_LINK 82-53] Creating IP database /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.cdb/xd_ip_db.xml
INFO: [SYSTEM_LINK 82-38] [11:04:09] build_xd_ip_db started: /trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/bin/build_xd_ip_db -ip_search 0  -sds-pf /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/emu/xilinx_u200_xdma_201830_2_emu.hpfm -clkid 0 -ip /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/iprepo/xilinx_com_hls_handlerEE_clES2_E3add_eCM5g51B_1_0,handlerEE_clES2_E3add_eCM5g51B -o /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.cdb/xd_ip_db.xml
INFO: [SYSTEM_LINK 82-37] [11:04:16] build_xd_ip_db finished successfully
Time (s): cpu = 00:00:08 ; elapsed = 00:00:07 . Memory (MB): peak = 1651.355 ; gain = 0.000 ; free physical = 339165 ; free virtual = 358363
INFO: [SYSTEM_LINK 82-51] Create system connectivity graph
INFO: [SYSTEM_LINK 82-102] Applying explicit connections to the system connectivity graph: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/cfgraph/cfgen_cfgraph.xml
INFO: [SYSTEM_LINK 82-38] [11:04:16] cfgen started: /trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/bin/cfgen  -sp handlerEE_clES2_E3add_eCM5g51B_1.m_axi_gmem:DDR[0] -sp handlerEE_clES2_E3add_eCM5g51B_1._arg_:DDR[0] -sp handlerEE_clES2_E3add_eCM5g51B_1._arg_4:DDR[0] -sp handlerEE_clES2_E3add_eCM5g51B_1._arg_9:DDR[0] -dmclkid 0 -r /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.cdb/xd_ip_db.xml -o /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/cfgraph/cfgen_cfgraph.xml
INFO: [CFGEN 83-0] Kernel Specs: 
INFO: [CFGEN 83-0]   kernel: handlerEE_clES2_E3add_eCM5g51B, num: 1  {handlerEE_clES2_E3add_eCM5g51B_1}
INFO: [CFGEN 83-0] Port Specs: 
INFO: [CFGEN 83-0]   kernel: handlerEE_clES2_E3add_eCM5g51B_1, k_port: m_axi_gmem, sptag: DDR[0]
INFO: [CFGEN 83-0]   kernel: handlerEE_clES2_E3add_eCM5g51B_1, k_port: _arg_, sptag: DDR[0]
INFO: [CFGEN 83-0]   kernel: handlerEE_clES2_E3add_eCM5g51B_1, k_port: _arg_4, sptag: DDR[0]
INFO: [CFGEN 83-0]   kernel: handlerEE_clES2_E3add_eCM5g51B_1, k_port: _arg_9, sptag: DDR[0]
INFO: [CFGEN 83-2228] Creating mapping for argument handlerEE_clES2_E3add_eCM5g51B_1._arg_ to DDR[0] for directive handlerEE_clES2_E3add_eCM5g51B_1.m_axi_gmem:DDR[0]
INFO: [CFGEN 83-2228] Creating mapping for argument handlerEE_clES2_E3add_eCM5g51B_1._arg_4 to DDR[0] for directive handlerEE_clES2_E3add_eCM5g51B_1.m_axi_gmem:DDR[0]
INFO: [CFGEN 83-2228] Creating mapping for argument handlerEE_clES2_E3add_eCM5g51B_1._arg_9 to DDR[0] for directive handlerEE_clES2_E3add_eCM5g51B_1.m_axi_gmem:DDR[0]
INFO: [SYSTEM_LINK 82-37] [11:04:20] cfgen finished successfully
Time (s): cpu = 00:00:04 ; elapsed = 00:00:04 . Memory (MB): peak = 1651.355 ; gain = 0.000 ; free physical = 339164 ; free virtual = 358362
INFO: [SYSTEM_LINK 82-52] Create top-level block diagram
INFO: [SYSTEM_LINK 82-38] [11:04:20] cf2bd started: /trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/bin/cf2bd  --linux --trace_buffer 1024 --input_file /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/cfgraph/cfgen_cfgraph.xml --ip_db /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.cdb/xd_ip_db.xml --cf_name dr --working_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.xsd --temp_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link --output_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int --target_bd emu/emu.bd
INFO: [CF2BD 82-31] Launching cf2xd: cf2xd -linux -trace-buffer 1024 -i /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/cfgraph/cfgen_cfgraph.xml -r /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.cdb/xd_ip_db.xml -o dr.xml
INFO: [CF2BD 82-28] cf2xd finished successfully
INFO: [CF2BD 82-31] Launching cf_xsd: cf_xsd -disable-address-gen -bd emu/emu.bd -dn dr -dp /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/sys_link/_sysl/.xsd
INFO: [CF2BD 82-28] cf_xsd finished successfully
INFO: [SYSTEM_LINK 82-37] [11:04:22] cf2bd finished successfully
Time (s): cpu = 00:00:02 ; elapsed = 00:00:02 . Memory (MB): peak = 1651.355 ; gain = 0.000 ; free physical = 339160 ; free virtual = 358362
INFO: [v++ 60-1441] [11:04:22] Run run_link: Step system_link: Completed
Time (s): cpu = 00:00:16 ; elapsed = 00:00:15 . Memory (MB): peak = 1670.371 ; gain = 0.000 ; free physical = 339199 ; free virtual = 358402
INFO: [v++ 60-1443] [11:04:22] Run run_link: Step cf2sw: Started
INFO: [v++ 60-1453] Command Line: cf2sw -sdsl /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/sdsl.dat -rtd /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/cf2sw.rtd -nofilter /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/cf2sw_full.rtd -xclbin /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/xclbin_orig.xml -o /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/xclbin_orig.1.xml
INFO: [v++ 60-1454] Run Directory: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/run_link
INFO: [v++ 60-1441] [11:04:25] Run run_link: Step cf2sw: Completed
Time (s): cpu = 00:00:02 ; elapsed = 00:00:02 . Memory (MB): peak = 1670.371 ; gain = 0.000 ; free physical = 339199 ; free virtual = 358402
INFO: [v++ 60-1443] [11:04:25] Run run_link: Step rtd2_system_diagram: Started
INFO: [v++ 60-1453] Command Line: rtd2SystemDiagram
INFO: [v++ 60-1454] Run Directory: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/run_link
INFO: [v++ 60-1441] [11:04:26] Run run_link: Step rtd2_system_diagram: Completed
Time (s): cpu = 00:00:00 ; elapsed = 00:00:02 . Memory (MB): peak = 1670.371 ; gain = 0.000 ; free physical = 338492 ; free virtual = 357695
INFO: [v++ 60-1443] [11:04:26] Run run_link: Step vpl: Started
INFO: [v++ 60-1453] Command Line: vpl -t hw_emu -f xilinx_u200_xdma_201830_2 --remote_ip_cache /home/stepha27/sycl-workspace/sycl/sycl/test/on-device/xocc/simple_tests/.ipcache -s --output_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int --log_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_log/link --report_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_report/link --config /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/vplConfig.ini -k /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/kernel_info.dat --webtalk_flag Vitis --temp_dir /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link --emulation_mode debug_waveform --no-info --iprepo /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/xo/ip_repo/xilinx_com_hls_handlerEE_clES2_E3add_eCM5g51B_1_0 --messageDb /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/run_link/vpl.pb /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/dr.bd.tcl
INFO: [v++ 60-1454] Run Directory: /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/run_link

****** vpl v2020.2 (64-bit)
  **** SW Build (by xbuild) on 2020-11-18-05:13:29
    ** Copyright 1986-2020 Xilinx, Inc. All Rights Reserved.

INFO: [VPL 60-839] Read in kernel information from file '/tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/int/kernel_info.dat'.
INFO: [VPL 74-74] Compiler Version string: 2020.2
INFO: [VPL 60-423]   Target device: xilinx_u200_xdma_201830_2
INFO: [VPL 60-1032] Extracting hardware platform to /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/vivado/vpl/.local/hw_platform
WARNING: /trinity/shared/pkg/devel/sdaccel/Vitis/2020.2/tps/lnx64/jre9.0.4 does not exist.
[11:04:50] Run vpl: Step create_project: Started
Creating Vivado project.
[11:04:53] Run vpl: Step create_project: Completed
[11:04:53] Run vpl: Step create_bd: Started
[11:05:37] Run vpl: Step create_bd: Completed
[11:05:37] Run vpl: Step update_bd: Started
[11:05:38] Run vpl: Step update_bd: Completed
[11:05:38] Run vpl: Step generate_target: Started
[11:05:55] Run vpl: Step generate_target: Completed
[11:05:55] Run vpl: Step config_hw_emu.gen_scripts: Started
[11:06:14] Run vpl: Step config_hw_emu.gen_scripts: Completed
[11:06:14] Run vpl: Step config_hw_emu.compile: Started
[11:06:18] Run vpl: Step config_hw_emu.compile: Failed
[11:06:22] Run vpl: FINISHED. Run Status: config_hw_emu.compile ERROR

===>The following messages were generated while processing /tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/vivado/vpl/prj/prj.sim/sim_1/behav_waveform/xsim :
ERROR: [VPL 43-3410] Failed to compile one of the generated C files.
Please recompile with "-mt off -v 1" switch to identify which design unit failed
.
ERROR: [VPL 60-773] In '/tmp/single_task_vector_add-47f9b65zz08txn/vxx_link_tmp/link/vivado/vpl/vivado.log', caught Tcl error:  xsc -c --gcc_compile_options -DBOOST_SYSTEM_NO_DEPRECATED --gcc_compile_options -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/simmodels/xsim/2020.2/lnx64/6.2.0/ext/protobuf/include --gcc_compile_options -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_embedded_scheduler_sw_0_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_embedded_scheduler_sw_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_1_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/src/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/src/cpp_src -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/src/sysc_src -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_xdma_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_xtlm_simple_intercon_0_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_xtlm_simple_intercon_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_1/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_1/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_2/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_2/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_3/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/ip/ip_3/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/bd_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_memory_subsystem_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_2_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/common -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/axi_crossbar/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/top -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/ddr_model/cpp_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim_tlm/ddr_model/systemc_srcs -I../../../../prj.ip_user_files/bd/emu/ip/emu_sim_ddr_3_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_0_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_0_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_1_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_1_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_2_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_2_0/sim -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_3_0/src -I../../../../prj.ip_user_files/bd/emu/ip/emu_icn_pass_3_0/sim -I../../../../prj.ip_user_files/bd/emu/sim -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/tps/boost_1_64_0 -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/simmodels/xsim/2020.2/lnx64/6.2.0/ext/protobuf/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/xtlm/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/xtlm_simple_interconnect_v1_0/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/common_cpp_v1_0/include -I/trinity/shared/pkg/devel/sdaccel/Vivado/2020.2/data/xsim/ip/emu_perf_common_v1_0/include -work xil_defaultlib -f emu_wrapper_xsc.prj
WARNING: [VPL 60-732] Link warning: No monitor points found for BD automation.
ERROR: [VPL 60-1328] Vpl run 'vpl' failed
ERROR: [VPL 60-806] Failed to finish platform linker
INFO: [v++ 60-1442] [11:06:22] Run run_link: Step vpl: Failed
Time (s): cpu = 00:01:32 ; elapsed = 00:01:56 . Memory (MB): peak = 1670.371 ; gain = 0.000 ; free physical = 338235 ; free virtual = 358341
ERROR: [v++ 60-661] v++ link run 'run_link' failed
ERROR: [v++ 60-626] Kernel link failed to complete
ERROR: [v++ 60-703] Failed to finish linking
INFO: [v++ 60-1653] Closing dispatch client.
Vitis linkage stage failed
Output /tmp/single_task_vector_add-47f9b65zz08txn/single_task_vector_add-47f9b6.xclbin was not properly produced by previous commands
clang-14: error: sycl-link-vxx command failed with exit code 255 (use -v to see invocation)
clang version 14.0.0 (https://github.com/triSYCL/sycl.git 0d2c6d5fca5e7822c8955955edc31e1acf74d4c0)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/stepha27/sycl-workspace/sycl-build/bin
clang-14: note: diagnostic msg: Error generating preprocessed source(s).

[j-stephan]

Okay, after diving into the log files produced by setting SYCL_VXX_KEEP_CLUTTER I was able to figure out that Vivado's own g++ executable is unable to find the correct path to its corresponding cc1plus. It simply wants to call cc1plus which isn't in my path. In my case it is located in /path/to/Vivado/2020.2/tps/lnx64/gcc-6.2.0/libexec/gcc/x86_64-pc-linux-gnu/6.2.0/. Adding this to my PATH lets the compile.sh script proceed. It feels a bit weird though that g++ is unable to deduce this by itself. Is this the right way to work around the problem?

[lforg37]

We usually (try to) run the tests only with the most recent available vitis version (that would be 2021.2 as of now). Can you try with it (that requires only 300 GB of disk space 😅) ?

If not, I can try to reproduce the bug and investigate it, but that will probably not be possible before in 3-4 weeks.

[j-stephan]

Okay, it appears that the Vitis tools are unhappy if I have another GCC compiler toolchain in PATH and LD_LIBRARY_PATH. So my workflow currently looks like this:

1. Load a modern GCC compiler to enable clang compilation.
2. Enable SYCL_VXX_KEEP_CLUTTER.
3. Use clang to compile the test case.
4. Wait until it crashes in the sycl-link-vxx step.
5. Unload the modern GCC version.
6. Navigate to the temporary directory mentioned in the log output and rerun the command found in <tempdir>/07-vxxlink.cmd.

[lforg37]

Ok we can try to reproduce these steps with our install to see if this bug still exists in recent vitis version.

[j-stephan]

Thanks! And I'll see what can be done about a more recent software stack. It also turned out that the node itself is still utilizing XRT 2019.1 which doesn't work well with more recent Vitis versions...

[keryell]

Try to use the most modern version of everything. :-)

[keryell]

We do not use the Vitis/Vivado setup because it breaks too many things, we just use the recipe from https://github.com/triSYCL/sycl/blob/sycl/unified/next/sycl/doc/GettingStartedXilinxFPGA.md#compiling-and-running-a-sycl-application
Are you using a similar setup?

[j-stephan]

So far I relied on the module system of the cluster to set up everything for me. I have now created a script that bypasses the module system for the Xilinx tools which looks very similar to the environment setup in the link. The results stays the same, though; vpl will fail in the config_hw_emu.compile step. When diving into the details /tmp/<build dir>/vxx_link_tmp/link/vivado/vpl/prj/prj.sim/sim_1/behav_waveform/xsim/compile.sh will fail with the following error message:

g++: error trying to exec 'cc1plus': execvp: No such file or directory

This error message goes away once I unload the gcc-11 module (which removes the entire gcc toolchain from PATH and LD_LIBRARY_PATH). Afterwards I can call the command from /tmp/<build dir>/07-vxxlink.cmd and it will happily continue (until the node's ancient xclbinutil complains about unknown parameters - I hope my IT department will resolve this soon).

[j-stephan]

IT installed Vitis 2021.2 on the cluster. The issue persists, though - if I load the gcc-11 module the Vitis tools will fail to detect their own cc1plus. My current work-around looks like this:

$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/trinity/shared/pkg/compiler/gcc/11.2.0/lib64 $SYCL_BIN_DIR/clang++ --gcc-toolchain=/trinity/shared/pkg/compiler/gcc/11.2.0 -std=c++20 -fsycl -fsycl-targets=fpga64_hls_hw_emu single_task_vector_add.cpp

This lets the HLS flow continue until xclbinutil is called.