Unable to merge PRs from remote branches because CI stalls

[metalstrolch]

When creating a branch on a non navit-gps repository, all CI-steps seem to pass and the branch will be shown "green" ci-wise.
When creating a pull request, the additional steps like codefactor are run as well and indicate green.
But the completion of "build_all" is never indicated. It stalls (with yellow circle) waiting for some event that doesn't seem to come. Note: it doesn't indicate failure in that state.

Currently #1007 is affected from that.

Can somebody with acces to CI configuration check what build_all is waiting for on pull request? There must be a difference to only branch, as the usual building of CI passes.

We had this situation already when we refactored the CI steps, but the config wasn't updated. Maybe it has something to do with requiring a signed apk, which cannot be built outside of navit-gps due to missing keys.

[jkoan]

I looked into it and it seems to be a issue with circleci.
The main cause is that we have our own circleci account where navit is build.
As an example the user who opened #1008 does not have a circleci account, so the build happens on navit-gps/navit on circleci. We both have a circleci account which leads to a situation where the workfow is started on the personal account and therefore it does not update the checks that should be updated.
My idea would be to scrap circleci and use the github actions feature, i think the integration from github actions to github checks will be way better that what we experience with circleci at the moment.

For all users of there own circleci account this dosn't mean that they need to stop using it, but for a pull request they only need to sync with upstream (what we already enforce)
I have already done some work on github actions, so i will open a PR soon. If everything is going well (after some tests with forks and such) i can also update the github check requirements

[metalstrolch]

I circumvent the problem by pushing the branches I want to merge directly to navit-gps instead of my metalstrolch fork. But most contributers can not. This is quite annoying at the moment. Btw, it would save navit-gps's CI account processing time if the contributers could use their fork's CI account to check. Thank's for caring.

[mvglasow]

@jkoan while I have no experience with Github Actions, I have recently implemented a few projects with Gitlab CI—might be worth revisiting a migration there ;-)

Some things would need to be looked at (such as deployment to Google Play or requiring a successful check for a merge requests). What I have successfully implemented there is custom CI images, based upon a generic one and extended with our build dependencies—somewhat speeds up builds and saves us CI time (the limit on Gitlab is 2000 minutes/month for a free plan).

[jkoan]

Github actions still in the works. Current Progress:
https://github.com/jkoan/test-navit/actions
So far only Sailfish and Wince are missing as well as doxygen and "merge into master" are untestet so far.
After both are done probably some optimizations will remain.
One "Problem" i see so far is that Artifacts will be stored within a zip from now on. Any opinions on this?

[metalstrolch]

If the results being zipped is the only problem then I'd not say this is a big deal. Currently getting the artefacts out of ci is a sophisticated task anyway as it requires you to have a real CI account. Having to unzip them: Never mind.
If your changes allow to share the CI build time amongst more user accounts speeds up things a lot.
Just my two cents.

[jkoan]

If your changes allow to share the CI build time amongst more user accounts speeds up things a lot.

That would not even be a Problem anymore since its free for public repos

[mvglasow]

Re Github Actions, I have just looked into them more deeply and they seem quite constrained. The folks over at Unison are considering moving to a different platform, mainly because of these limitations. Most importantly, there doesn’t seem to be a way to run your build on any Docker image of your choice, but only a few available images.

@jkoan you started a mirror repository on GitLab back when Microsoft acquired Github. I might have suggested this in the past, but maybe it’s worth reconsidering for CI? GitLab even has instructions on how to migrate from CircleCI to GitLab CI. All in all, the structure is not too different, and we don’t seem to be making heavy use of the CI’s features (most is done in the form of shell commands) – which helps with portability.

[jkoan]

Regarding the "constrained" nature of GitHub actions I need to disagree. But this could also be a thing of evolution.
As I know (and have tested) you get a temporary vm with a base system of Ubuntu, Mac or Windows (all in multiple versions and a lot of preinstalled software) and use it directly or through there docker image support. With this you can execute any image you want from any registry you want. This is what my draft GitHub actions or does as well.

Regarding moving to gitlab: I don't want to introduce yet another shift. Even when it's owned by Microsoft now unless we need to pay or there are other important reasons I would like to keep the status quo.

Despite having said that I definitely want to keep everything as portable as possible. I want to even make it more portable than now.
Changes I am planning are to reduce the build scripts to the bare minimum (configure and make -> cmake and make) and reduce the clutter of the dependency installation. For the later I am planning to move the build to purpose build images with all dependencies installed already. This will also help with build times as all software is already installed.

By using container images and the bare minimum of scripts together with the power of the build system itself (cmake/gradle) I think we are already really portable and future proof in ci regards.