GitLab + Flatpak – GNOME’s full flow

In this post I will explain how GitLab, CI, Flatpak and GNOME apps come together into, in my opinion, a dream-come-true full flow for GNOME, a proposal to be implemented by all GNOME apps.

Needless to say I enjoy seeing a plan that involves several moving pieces from different initiatives and people being put together into something bigger, I definitely had a good time ✌️.

Generated Flatpak for every work in progress

The biggest news: From now on designers, testers and people with curiosity can install any work in progress (a.k.a ‘merge request‘) in an automated way with a simple click and a few minutes. With the integrated GitLab CI now we generate a Flatpak file for every merge request in Nautilus!

In case you are not familiar with Flatpak, this technology allows anyone using different Linux distributions to install an application that will use exactly the same environment as the developers are using, providing a seamless synchronized experience.

For example, do you want to try out the recent work done by Nikita that makes Nautilus views distribute the space between icons? Simply click here or download the artifacts of any merge request pipeline. It’s also possible to browse other artifacts, like build and test logs:

artifacts.png

Notes: Due to a recent bug in Software you might need to install the 3.28 Flatpak Platform & Sdk manually; this usually happen automatically. In the meantime install the current master development Flatpak Nautilus with a single click here. In Ubuntu you might need to install Flatpak first.

Parallel installation

Now, a way to quickly test latest works in progress in Nautilus it’s a considerable improvement, but a user probably don’t want to mess up with the system installation of Nautilus or other GNOME projects installation, specially since it’s a system component. So we have worked on a way to make possible a full parallel installation and full parallel run of Nautilus versions alongside the system installation. We have also provided support for this setup in the UI to make it easily recognizable and ensure the user is not confused about what version of Nautilus is looking at. This is how it looks after installing any of the Flatpak files mentioned above:

Screenshot from 2018-03-19 21-35-41.png

We can see Nautilus system installation and the developer preview running at the same time, the unstable version has a blue color in the header bar and a icon with gears. As a side note you can also see the work of Nikita I mentioned before, the developer version of the views now distribute the space of the icons.

It’s possible to install more versions and run them all at the same time, you can see here how the different installed versions are found in the search of GNOME Shell where I also have the stable Flatpak Nautilus installed:

Screenshot from 2018-03-19 21-37-24

Another positive note is that this also removes the need to close the system instance of the app when contributing to GNOME, it was one of the most reported confusing steps of our newcomers guide.

Issues templates

One of the biggest difficulties we have for people reporting issues is that they either have an outdated application, the application is modified downstream, or the environment is completely different as the one the developers are using, making the experience difficult and frustrating for both the reporter and the developer.  Needless to say all of us had to deal with ‘worksforme‘ issues…

With Flatpak, GitLab and the work explained before we can fix this and boost considerably our success with bugs.

We have created a “bug” template where reporters are instructed to download the Flatpaked application in order to test and reproduce in the exact same environment and version as the developers, testers, and everyone involved is using. Here’s part of how the issue template looks like:

bug.png

When created, the issue renders as:

preview.png

Which is considerably clearer.

Notes: The plan is to provide the stable app Flatpak too.

Full continuous integration

The last step to close this plan is to make sure that GNOME projects build in all the major distributors. After all, most of us are working both upstream in GNOME and downstream in a Linux distribution. For that, we have made a full array of builds that runs weekly:

pipelines

Which also fixes another issue we have experience for years: Distribution packagers delivering some GNOME applications different than intended, causing subtle but also sometimes major issues. Now we can point out to this graph that contains the commands to build the application as exact documentation on how to package GNOME projects, directly from the maintainer.

‘How to’ for GNOME maintainers

For the full CI and Flatpak file generation take a look at Nautilus GitLab CI. For the cross distro weekly array additionally create an scheduled pipeline like this. It’s also possible to do more regularly the weekly array of CI, however keep in mind the resources are limited and that the important part is that every MR is buildable and that the tests passes. Otherwise it can be confusing to contributors if the pipeline is failing for a one of the jobs and not for others. For non apps projects, you can pick a single distribution you are comfortable with, other ideas are welcome.

A more complex CI is possible, take a look at the magic work of Jordan Petridis in librsvg. I heard Jordan will do a blog post soon about more CI magic, which will be interesting to read.

For parallel installation, it’s mainly this MR for master and this commit for the stable version; however there has been a couple of commits on top of each, follow them up to today’s date (19-03-2018).

For issues templates, take a look at the templates folder. We were discussing here a  default template to be used for GNOME projects,however there was not much input in there so for now I imagined better to experiment with this in Nautilus. Also, this will make more sense once we can put a default template in place, this is something GitLab will probably work on soon.

Finishing up…

On the last 4 days Ernestas Kulik, Jordan Petridis, and me have been working trying to time box this effort and come with a complete proposal by today, each of us working in a part of the plan, and I think we can say we achieved it. Alex Larsson and other people around in #flatpak provided us with valuable help. Work by Florian Mullner and Christian Hergert were an inspiration for us too. Andrea Veri and Javier Jardon put a considerable amount of their time into setting up an AWS instance for CI so we can have fast builds. Big thanks to all of them.

As you may guess, this CI setup for an organization like GNOME with more than 500 projects is quite resource consuming. Good news is that we have some help from sponsors happening, many thanks to them! Stay tuned for the announcements.

Hope you like the direction GNOME is going, for me it’s exciting to modernize and make more dynamic how GNOME development happens, I can see we have come a long way since a year ago. If you have any thoughts, comments or ideas let any of us know!

Advertisements

6 thoughts on “GitLab + Flatpak – GNOME’s full flow

  1. Impressive work! Really like how simple you’ve made it testing software after a merge. Not everyone are developers with a environment set up or understand or care for it, but now a lot more users will be able to provide valuable feedback.

    Pssst….
    Now with the new friendship, if you can only convince Canonical to switch to GitLab. I’ve done some work on Launchpad, but jeez it feels so old compared to GitLab now. Glad you did the move!

  2. HI,

    Kudos for the great work. The `.gitlab-ci.yml` looks great.

    `fedora:master:` take 15m that looks long. Do you have plan for unit test to speed feedback and check more corne cases ?

    Regards,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s