Progress output of COMSOL

[laigner]

Hi,
first of all thanks for mph it is great. And now to my question.
Is there a smooth way to get the progress information of COMSOL during the solvation of the model? Like e.g. the progress bar or messages about the current calculation? Would be awesome if I can print it somewhere or illustrate it in my pyqt5 application.

Thanks and best regards,

Lukas

[john-hen]

Hi Lukas, that's a great question. I don't know the answer, but would also be interested to know. So maybe we can make some, um… progress here.

The Programming Manual of Comsol 5.6 has a section "Handling of Progress Information" on page 917. It mentions some classes which are further explained in section "GUI Classes" on page 925. I can't find those classes in the API reference, but ProgressContext for example is defined in the com.comsol.model.util Java name space and can be inspected like so:

>>> import mph
>>> mph.start()
>>> import com.comsol
>>> dir(com.comsol.model.util.ProgressContext)
['__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__',
'__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__',
'__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'cancel',
'cancelButtonVisibilityUpdated', 'equals', 'finished', 'getClass',
'handleEvent', 'handleProgressError', 'hashCode', 'isCanceled',
'isStopped', 'notify', 'notifyAll', 'progressDescriptionUpdated',
'progressLabelUpdated', 'progressLogUpdated', 'progressUpdated',
'reset', 'setStoppable', 'started', 'stop', 'stopButtonVisibilityUpdated',
'toString', 'wait']

Some of the method names look promising, but I haven't, at this point, looked into what they do at all. It just may be an avenue to pursue.

The Programming Manual also explains how to display the progress window as either a Swing or SWT widget. I don't know if this even works from Python (via JPype). I do remember that the Matlab LiveLink has a way of displaying these progress windows as simulations are running. So maybe it is possible from Python as well.

However, that's not really what you want here. You don't want Swing or SWT, you have your own GUI framework, Qt. You want access to the progress information only, so you can display it yourself. This could mean registering a callback function, which gets called whenever there is a change in progress, or keep polling the progress status from a separate thread, which could then send a (Qt) signal to the main thread so that the GUI can trigger an update.

One thing that should already work, but is a little hacky, is diverting the log to a file, then read it back from disk and parse out the progress information. You can create the log file like so:

import mph
client = mph.start()
model = client.load('capacitor.mph')
client.java.showProgress('log.txt')
model.solve()

This puts lines containing strings such as Current Progress: 2 % in that log file (which maybe should have a unique name and be placed in a temporary directory). You could parse the file in a separate QThread and then send the signals. But it's not so clear how to do the parsing. "Current progress" only refers to the current step, which could be the geometry sequence, or the mesh, or any of the solver sequences. So it's not straightforward to get the overall progress, and using the ProgressContext class (or another class Comsol may provide) might prove to be the better approach.

[john-hen]

So it looks like creating and parsing that log file is the best option after all, whereas using Comsol's ProgressContext class, provided for this very purpose, is quite inconvenient and cannot be done in Python alone.

Here is why… The Java class ProgressContext provides five different methods, such as progressUpdated(double progress), that we would need to override, as their default implementation does nothing. See page 925 in the Programming Manual of Comsol 5.6. But extending a Java class (from within Python) is not (currently) possible with JPype. See jpype-project/jpype#420.

What we would like to do is something like this:

import mph
from jpype import JImplementationFor, JImplements, JOverride


@JImplementationFor('com.comsol.model.util.ProgressContext')
class _ProgressContext:

    @JOverride
    def started(self):
        print('Starting step.')

    @JOverride
    def progressUpdated(self, value):
        print(f'current progress: {round(value*100)}%')

    @JOverride
    def progressDescriptionUpdated(self, text):
        print(f'current step: {text}')

    @JOverride
    def progressLogUpdated(self, message):
        print(f'log message: {message}')

    @JOverride
    def finished(self, t):
        print('Finished step.')


@JImplements('java.lang.Runnable', deferred=True)
class Runner:

    @JOverride
    def run(self):
        model.solve()


client = mph.start()
model = client.load('capacitor.mph')
import com.comsol
worker = com.comsol.model.util.ProgressWorker()
context = com.comsol.model.util.ProgressContext()
worker.setContext(context)
runner = Runner()
worker.run(runner)

Already this is a bit inconvenient. To get the progress information sent out, we have to use Comsol's ProgressWorker. But we can't just tell it to call a Python function like model.solve, as we would like to, it needs to be wrapped in a Java "runnable", as that's the argument type the worker expects. And to process the progress information, we have to modify the ProgressContext instance that receives it. However, the above code does not work. It only customizes the Python wrapper of that class. (We could call, for example, context.started() at the end to see that that's true.) But it doesn't actually change the Java class that is running inside the Java virtual machine. So it keeps doing nothing.

The Java class can only be extended with Java code. Something like this:

import com.comsol.model.util.ProgressContext;

public class ProgressContextExtension extends ProgressContext
{
    public void started() {
        System.out.println("Starting step.");
    }

    public void progressUpdated(double progress) {
        System.out.println(progress);
    }

    public void progressDescriptionUpdated(String description) {
        System.out.println(description);
    }

    public void progressLogUpdated(String message) {
        System.out.println(message);
    }

    public void finished(Throwable t) {
        System.out.println("Finished step.");
    }
}

Or rather, we'd have the class hold on to the status information it receives so that we can then query it from Python. That Java class needs to be compiled, with a JDK that is not newer than the one Comsol uses. Then put into a .jar file, which we can then load with JPype to pass the extended class instance to the worker. This is all highly inconvenient. (And, to be clear, I have no intention of adding any Java code to the library.) Maybe JPype will eventually support a variation of the first approach. But that's arguably quite tricky. After all, it would need to compile Java code "live", within the running Python session.

Using the Swing widgets should be possible, the JPype documentation has a section on that. But I haven't had the patience to actually try that out. It's certainly a hassle as well, if the Java demo in the Programming Manual (page 917) is any indication.

In light of all that, just calling client.java.showProgress('log.txt') and then parsing that log file is the much easier option. Python is good at parsing text.

[laigner]

Thanks again for your effort and the great answer. Just to let you know my current workaround:
Using the python module watchdog to monitoring the file changes in the log.txt, created by client.java.showProgress('log.txt').
Passing at every change the current progress line in the log.txt to the pyqt5 interface.

[Magaud59]

Dear Lukas,
you've been actually looking a the problem I'm trying to solve right now... thanks for that.
I guess I'll make use of the trick you've been setting up...
Just one question maybe... where is located the log.txt file that you are talking of... I'm under win10 and I've got a hard time to find out...

Thanks by advance.

Mat

[laigner]

Hello Mat,

its long ago I used comsol and the mph package, so I am not sure about that. I would guess if you use client.java.showProgress('log.txt') it should take the current working directory to save the data. If you didn't specify this, it should be the path from where you calling the client?
Also I guess that there is the possibility to give a full path as an argument like client.java.showProgress('/path/to/directory/log.txt').

But as I said I am not familiar anymore with this package and comsol. Maybe you can try it and response if it doesn't work.

Best regards,

Lukas

[Magaud59]

Thanks for your prompt feedback.
I work indeed pretty smooth adding that command.
Now, I just have to dig into watchdog... as usual, I've got no clue what it is about but I'll do it ;) ... and normally, should come out fine.
I'm wishing you and excellent day.

With regards

M