The new contribution workflow for GNOME

Hello community,

I have big good news to share with you. You might know we have been working for years on materializing what we wanted the future of contribution to be, we did multiple iterations and we worked full time on our developer experience… and finally, I’m glad to announce, we achieved it, we have a new way to contribute to GNOME!

One image says more than 1000 words, the whole process of contributing to GNOME is as easy as you will see, all documented in the new newcomers wiki

builderstep2step3

No specific distribution required. No specific version required. No dependencies hell. Reproducible, if it builds for me it will build for you. All with an UI and integrated, no terminal required. Less than five minutes of downloading plus building and you are contributing.

Can you imagine how this changes the GNOME contribution story? We went from requiring either latest Fedora or Ubuntu, fighting dependencies and random issues, taking more than 80 modules to build just for contributing to a single app. It was a pain.

As an example, Nautilus with the previous tool and workflow took around 6 hours the first time if no issues were present. Now it’s 5 minutes, with no possible build issues (forgive exceptions in the rule 🙂 ).

I think we just opened a new world for contributors.

The work behind it

Of course, a change as big as this didn’t come overnight, this is possible because GNOME and sponsors put the time and resources on it, with rock-stars like Alex Larsson creating Flatpak and Christian Hergert creating Builder, working both for years nonstop in these technologies, with no short term benefit.

Finally the benefit is here, the future we imagined and shaped 5 years ago is coming together, and it’s shining.

Thanks a lot to the people involved, also specially Bastian Ilso for his guidance, design and writing of the new wiki guide.

Hope you enjoy all the work we did, I’m looking forward for your feedback and to fix the issues you may find (contact us in IRC in #newcomers). And soon, to have your first contribution with GNOME done 🙂

 

PD: Please follow the newcomers wiki to have it working, lot of work to make this happen was done in Flatpak 0.9.1, when Ubuntu 16.04 has 0.8.4 for now, so we say to use a PPA for have it updated. I tested thoughtfully in Ubuntu 16.04 and Fedora 25, and it works out of the box following the wiki making sure Flatpak is updated. Thanks all for the feedback so far! 🙂

PD2: I just realized I had a small error when doing the switch to the new wiki and the instructions for Ubuntu 16.04 and PPA got lost. Now it’s fixed, try again and tell us how it goes! 🙂

PD3: Cool video of Jono Bacon showing what Endless does with the same technology https://twitter.com/jonobacon/status/817059475437879305

Advertisements

38 thoughts on “The new contribution workflow for GNOME

  1. Looks very interesting. Now that Canonical have decided to drop Unity in favour of Gnome it’s time for Gnome devs to really up their game and prove it’s a worthy replacement.

  2. Hey this is great… but… I just tried it in F25 and F24 without luck… In F24 there is no support for flatpakref in the included version of flatpak and in F25 I got a not Found Sdk error…

    1. Hey, is mentioned in there that you need Software 3.22, otherwise to execute the command line there. I tried in f24 and ubuntu 16.04 and it works. Make sure your flatpak is updated (version 0.9.1) with sudo dnf update flatpak.

  3. Haven’t had a chance to try this out but if it works as advertised this is really quite the milestone – congrats!
    I would really love to check what would be needed to set something like this up for Xfce (we’ve been working towards automated GUI testing with Docker and behave lately).

    1. Cool, it doesn’t matter if its xfce or not. If the app uses Flatpak, you can follow what’s written in there. Even if it’s a guide for GNOME contribution, it works for any app using Flatpak (not needed to be a GNOME app at all). Just that I put extra effort on the Newcomers apps myself.
      If you find any issues, please tell us!

  4. This is totally awesome.. I noticed that I had to install ostree and flatpak-builder for gnome-builder to be able to build nautilus. So probably there is some work to do to get those running inside the gnome-builder flatpak..

    Randomly, a debugger would be neat, I got nautilus building, but it wouldn’t run 🙂 hehe
    Granted I’m on Ubuntu 16.04, I’m sure things will get solid…

    1. Hey, thanks for your feedback! Seems you didn’t follow yhe neecomers guide, we request a ppa to use flatpak 0.9.1 instead of the one on ubuntu 16.04, lot of work to make this happen was done in that version, no surprise it didnt work for you 😛 Ostree and flatpak-builder are installed automatically, I tested througfully in ubuntu 16.04 before making this 🙂 Try again following the wiki and tell me how it goes!

      1. Ha, just realized I had a small error when doing the switch to the new wiki and the Ubuntu instructions to use a PPA for an updated Flatpak were lost 😦 now it’s fixed, please try again and tell me how it goes! 🙂

    1. We can definitely have a copy on the website, we have the setup already to have this kind of mirrors, so good idea, I’ll take a look at that.
      We will continue to use the wiki as source though, it’s easy to modify and anyon can contributez which helps in this case.

      1. That would be very good. One of my main frustrations as a newcomer to the GNOME developer world is that the documentation is spread all over, and I still have no idea where to start. Where do GNOME devs expect me to learn GTK+? There are two or three different places to go for a documentation (one from GTK page, one from GNOME and one for my language of choice bindings). What is the right one?

        This kind of guide makes it much easier for getting started (thank you so much for that), but for coding itself, I still feel kind of lost.

      2. Right. So if I understand correctly, you don’t see problems with getting started anymore, now the only problem is learning development with gtk+ right? We link to it (to one place) from the newcomers guide, is that not enough? Are you confused because you search on internet and diferent places come up? When you say documentatioj for your binding, you mean that’s not the one and only documentation you need to look at? What language?

  5. Sometimes an application developer needs to test the application against a specific version/commit of a library. For example to track down a regression, doing a git bisect on the library.

    With Jhbuild we have a total control over which version of each module is installed. `jhbuild list gtk+ | wc -l` returns me 15 modules to build, a lot of apps don’t require much more dependencies than that. When I started to contribute to GNOME I don’t remember having a lot of difficulties learning Jhbuild.

    Anyway, I think it is a good news to make it easier to start contributing, but I’m a bit skeptic about that approach to work in all cases, especially when there is a problem.

    Also, an IDE hides some stuff that is done in the background. Again, in case of problems, the new contributor will not know what to do. With a terminal, when we launch a command, we know what happens, we have more control.

    An additional hurdle is that GLib/GTK+ has a steep learning curve. Once a contributor has built the code, he/she is far from ready because he/she will understand nothing about the code. A book needs to be written and kept up-to-date, that’s what I’m doing here:
    https://people.gnome.org/~swilmet/glib-gtk-book/

    In my opinion a good GNOME contributor is someone who has already written a GTK+ app him/herself before.

    1. “Sometimes an application developer needs to test the application against a specific version/commit of a library”
      Afaik this is being worked on. In any case, the initial target audience is newcomers. A non newcomer won’t have any problem building with JHBuild if necessary for this. Of course the long-term goal is to have it for as much as we can.

      “When I started to contribute to GNOME I don’t remember having a lot of difficulties learning Jhbuild.”
      Yeah, I’m not looking if I have problem or not, I’m looking if others have problems or not 🙂 They do.

      ” I think it is a good news to make it easier to start contributing, but I’m a bit skeptic about that approach to work in all cases, especially when there is a problem.”
      Indeed, neither does JHBuild. It’s about trade-offs, and in my opinion and what I gathered as a general consensus is that this is much much better.

      “Also, an IDE hides some stuff that is done in the background”
      Not sure how that’s bad, you know that most of developers around there use IDE’s for working right? Universities etc. use them. Of course this is probably different for our already established contributors like you. For that, we also include CLI instructions in the wiki (added yesterday, not present in the pics in here).

      “Again, in case of problems, the new contributor will not know what to do. With a terminal, when we launch a command, we know what happens, we have more control.”
      Not really, that’s why the come to #newcomers. They didn’t know with JHBuild either, and had far more many problems and gotchas.

      “An additional hurdle is that GLib/GTK+ has a steep learning curve. Once a contributor has built the code, he/she is far from ready because he/she will understand nothing about the code”
      Indeed, I couldn’t agree more. Not sure how this is related to building with Flatpak + Builder or JHBuild though.

      “A book needs to be written and kept up-to-date, that’s what I’m doing here:
      https://people.gnome.org/~swilmet/glib-gtk-book/
      I always aplauded your work in this, I’m more than aware. Keep in mind though that the current standard is not “read this whole book”, but rather the opposite. Small tutorials and contributing on the way. Either we adapt to the mainstream standard or we are missing 90% of the possible contributors.

      “In my opinion a good GNOME contributor is someone who has already written a GTK+ app him/herself before.”
      A established contributor, maybe. A newcomer, no. And of course we shouldn’t require that to contribute to GNOME. As a matter of curiosity, I wrote my first gtk+ app when I was already one year maintainer of Nautilus 🙂

      1. If the new way to contribute works well, and if you don’t have too many questions to answer on IRC/mailing lists, then that’s great.

        From the blog post: “Less than five minutes of downloading plus building and you are contributing.”
        That’s true only if the contributor already knows well GTK+. From my experience, newcomers who know nothing about GTK+ write horrible patches with indentation problems and such like, and it takes a huge amount of time for the reviewer to explain everything. That doesn’t scale, a book scales. With the GSoC, there is an assigned mentor to guide and review patches of one or a few contributors. But a maintainer cannot do that for all potential contributors (and with a lower barrier to entry there will be more of them), unless there is a very good documentation and the contributors are autonomous (which generally means that they have already written a GTK+ app before).

        The fact that a lot of developers use IDEs doesn’t mean that’s a superior way. On Linux I think we can assume a developer to be able to use the terminal, and with a terminal I think we better know what the computer does and how things work. But that’s just my opinion.

        > Keep in mind though that the current standard is not “read this whole book”, but rather the opposite. Small tutorials and contributing on the way.

        Lol, small tutorials and contributing on the way means that the reviewers will need to re-explain and guide contributors all the time. That doesn’t scale. And the developers who want to develop their own GTK+ app will write horrible code with a horrible code architecture. GLib/GTK+ is a whole development platform, it’s not a small JavaScript library that can be learned with a small tutorial. If reading more than 5 pages is too much for a computer scientist, I do not have much esteem for that person.

        I’ve read a book recently (Facts and fallacies of software engineering, Robert Glass) where it was explained : “The wisdom of the software field is not increasing. With the explosion of newcomers arriving in this fast-growing profession, the increasing experience level of the growing-olders is more than overcome by the low experience level of the hordes of newbies.”

  6. “That’s true only if the contributor already knows well GTK+. That’s true only if the contributor already knows well GTK+. ”
    I made a wrong choice of words then, what I mean is “ready to contribute for first time” in that case. A big part in this process is to have the app built.

    “From my experience, newcomers who know nothing about GTK+ write horrible patches with indentation problems and such like, and it takes a huge amount of time for the reviewer to explain everything.”
    Exactly, that’s the reason newcomers initiative exists! It’s an initiative where maintainers opt-in. So people/maintainers/reviewers that doesn’t want to deal with what you mention, don’t do it. We try to redirect them to where is appropriate. The good thing is that eventually, those maintainers will also get benefited from a more experienced contributor that went through others maintainers that are willing to deal with the learning process.

    “The fact that a lot of developers use IDEs doesn’t mean that’s a superior way.”
    I think you are missing the point here. We guide people for the first times. Not forever. You are free to use whatever you want, including other IDE or editors, and we point that out in the first sentence of the BuildProject wiki page.

    “Lol, small tutorials and contributing on the way means that the reviewers will need to re-explain and guide contributors all the time. ”
    I think you are missing my point here too. This is not your (my/ours) call. This is how it is, the standard are not set by us. Either we adapt or we miss them. I don’t have a source to back this up, but I think is well known video tutorials, stackoverflow, small tutorials like the ones in rust-lang.org and “games” like freecodecamp.com are the standard nowadays.
    Would I prefer someone that have read a books of gtk+ and has written an app and is maintainer already to contribute? Of course, I’m extremely happy when that happens, and they get recognized instantly in the whole organization (e.g. Georges, Giovanni…you know them). Do I think we should not provide a way for all the other 99.99% potential contributors? No I don’t, because if that would be the case, I wouldn’t be in GNOME in the first place.

    “If reading more than 5 pages is too much for a computer scientist, I do not have much esteem for that person.”
    That’s fine, that’s why you didn’t opt-in in the newcomers initiative. That’s my main reason, I don’t want our maintainers to feel they have to put this weight on them if they don’t want, and at the same time, provide a way for those newcomers and maintainers that do. It’s about redirection.

    “The wisdom of the software field is not increasing. With the explosion of newcomers arriving in this fast-growing profession, the increasing experience level of the growing-olders is more than overcome by the low experience level of the hordes of newbies.”
    This is a good point, I actually though in length about this, because we can see this in programs like GSoC. You might thing is better in GSoC, but it’s not, and that sometimes frustrates the mentor/maintainers.
    I’m trying to minimize this on the GSoC admin side (higher the bar, some help to mentors, some reward to mentors for their effort, etc.), but I’m also trying to minimize it on the technical side, by providing enough documentation in the wiki to “just point to them”, or point how they can search for a specific part of the documentation. This way is smoother than requiring reading a whole book before starting, because they can learn in the way and with practical output.
    Where I can see a book to take place is in the “next step”, and for people like me it’s very welcomed.

    1. I just realized some thing would be better to clarify.

      It looks on some points that I’m excluding you or your way from the initiative, and that your vision has no place in there. That’s not what I want. But I think we have to accept how’s the current reality and standards, and decide if we want them. For what I can see you prefer to higher the bar and standards, but then I’m not sure that is how we should move the newcomers initiative. I see the newcomers initiative more an smooth as possible entrance from a different world (the one we cannot control: GitHub contribution, University knowledge) to ours. Once inside, you can decide whether to go further or not, and there we can raise our standards if wanted. But we should strive to have at least a set of places where they can blend into GNOME as smooth as possible.

      Also clarifying on
      “If reading more than 5 pages is too much for a computer scientist, I do not have much esteem for that person.”
      I see where you are coming from, I agree. Sometimes I feel that if you don’t take the time to even read a simple wiki, then idk :/
      However, I think there are a set of people who just work differently, and prefer and are used to the new standards around there, like I mentioned freecodecamp, rust-lang.org etc. where you have feedback about your work instantly.
      So I think the best way to go is, provide this feedback while you are learning, and after this is done, *then* yes, for some things you might say “for this you need to read more…here is the part you need to read” or “this project/task requires deep knowledge of $whatever, better to get used and read about $whatever”. I think most of our newcomers tasks doesn’t need this deep knowledge though.

      Anyway, I’m happy to keep discussing and clarifying, is a good chat

  7. Nice. Especially since im interested in Gtk and Nautilus 🙂
    I’m not a fan ob Flatpack/Sandboxes, but in this case it looks useful.

  8. Been waiting for this for years ! I remember those night looking at jhbuild output and trying to workaround issues completely unrelated to what I wanted to do. I hope a new generatio of contributors emerges now that contributing can be that simple.

    1. It took me two weeks to make my first contribution four years ago becaude of this :/ (Granted, it was gnome-shell, but still) So yeah, I’m so happy we changed that now to the other extreme!

    1. And it does, that’s the point of using ostree, etc. in Flatpak no? :/ Or did I miss something?

      As a matter of curiosity, there was an issue because of this, ostree resets the mtime to 0 of all files for the reproducible part, and with ninja build system was making every build from scratch because it was detecting like the file was modified (mtime != previous mtime), which from some time until we fixed it in Flatpak was making the builds of ninja inside Flatpak extremely slow.

      So yeah, idk what you mean you wouldn’t call it reproducible. What in Flatpak do you think doesn’t abide to reproducible builds?

  9. I think it’s time to have a chat with Canonical to decide which idea can be better used in this world, snaps or flatpaks. Which are easier for developers, easier to maintain (both, development of snap and flatkpak installer source code; and packages of snap and flatpak format). Which one can have wider usage (integration with all different DE-s, usage in docker?) and finally, license.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s