`dace.Config` is not Thread Safe and a Globally Shared Object

[philip-paul-mueller]

I am pretty sure that all of us are aware that dace.Config is not thread safe.
However, I think that this aspect has been overlooked until now and its impact was not realized, as we all thought, that this does not happen since we will not use DaCe in a multi threaded context.
Today I realized that this is not true as we run the tests in parallel which means that DaCe is affected by this issues:

Consider the following function:

def my_awesome_optimizer(sdfg, optimizer_options) -> dace.CompiledSDFG:
    with dace.config.temporary_config():
        configure_dace_for_my_awesome_optimizer()
        return _my_awesome_optimizer_impl(sdfg, optimizer_options)

Now consider two thread and the following scenario:

• Thread 1 enters my_awesome_optimizer() which will create a new "configuration context".
  This means it will safe the old configuration and store it inside a file, for the sake of argument, let's assume that this are the default settings.
• Thread 1 will then enter the implementation of the optimizer and start doing stuff, which will take some time.
• Thread 2 will also enter my_awesome_optimizer() and also create a new configuration context.
  However, it will store the configuration that was modified by thread 1 to disc.
• Thread 2 will then also enter the implementation of the optimizer.
• Thread 1 has finished the optimization process and exits the with clause, this will trigger the restoration of the stored configuration.
  This means that now the default settings have been restored and have been activated.
• From this point on thread 2 will no longer use the configuration that was established by `configure_dace_for_my_awesome_optimizer(), instead it will use the default configuration.

Thus it is quite clear that there are issues and that they are present inside the tests.

Another implication from the fact that dace.Config is a globally shared object, is that it actually limits what you can set.
Consider the following:

def _compile_with_different_opt_level(sdfg, opt_level):
    with dace.config.temporary_config():
        dace.Config.set("optimization_level", value=opt_level)
        return sdfg.compile()

@pytest.mar.parameterize("opt_level_and_timing", [("-O2", 1.0), ("-O3", 0.5)])
def test_compile_opti(opt_level_and_timing):
    opt_level, timing = opt_level_and_timing
    sdfg = get_sdfg()
    csdfg = _compile_with_different_opt_level(sdfg, opt_level)
    mean_runtime = timeit.time(csdfg)
    assert mean_runtime < timing

With a similar scenario it is kind of arbitrary which compiler flags are used.

The solution would be to turn the configuration object into thread local data.

The bug is present in current main: a214529