[Nix-dev] Some bug tracker experience and RFC on improvements
Eric Seidel
eric at seidel.io
Mon Mar 16 03:10:54 CET 2015
Hi Austin,
Thanks for putting together such a detailed list of suggestions! I
particularly like the suggestion to have a list of maintainers for
specific subsystems. This is especially important in light of the work
we've been doing lately with nix-on-darwin, as the package maintainer
may not even use darwin and thus be unable to help resolve
darwin-specific issues.
I'll happily volunteer to co-maintain darwin support as I rely on nix to
build my whole development environment.
Eric
On Sun, Mar 15, 2015, at 16:43, Austin Seipp wrote:
> Hi *,
>
> Over the past few days, while I've been waiting for things to compile,
> I've been casually triaging the NixOS pull request list and the actual
> bug list, adding labels to everything to categorize issues. I think
> all of the PRs open as of today are labeled, which is great. The first
> few pages of the bug list are also almost labeled, but I haven't gone
> much farther.
>
> I want to take a moment emphasize the importance of doing this, and
> some thoughts on improving it, based on my experience doing similar
> things for other projects. Please note, I'm talking almost exclusively
> about the 'nixpkgs' repository.
>
> As many of you may be aware of by now, GitHub has an issue tracker.
> It's integrated with the Pull Request capability. It's relatively
> simple by design. But this has some knock-on consequences - it has a
> relatively anemic ability to extend metadata (e.g. custom issue
> fields), and personally I've found the full-text search capabilities
> to be relatively poor. Their issue tracker is quite simple like I
> said, but what this means is that metadata is vitally important to
> triaging bugs and having maintainers get to them easily and in a
> timely manner.
>
> For example, I'm generally interested in all kernel bugs, and things
> like security bugs and issues so I can get CVEs fixed (I know, I
> haven't had much time for this recently.) But I only added the
> 'kernel' label just on Friday, I think!
>
> Also, I want to know about all the bugs for the 15.05 release that
> need to be fixed, which are kernel/systemd bugs. And the ones that are
> critical priority, that haven't been fixed. And the ones that have
> been fixed, so we can add to the release notes. That way I know what
> to do about them.
>
> And, as I'm sure most of you are aware, GitHub's notification system
> is not very lean. If you 'watch' a repository, you get spammed for
> every update to it! This works OK for small projects where I do
> everything - it does not scale very nicely to more people, meaning in
> practice I have a GitHub filter in my email with hundreds of unread
> things for NixOS subsystems I don't maintain.
>
> All of this makes the need for precise metadata very important. It is
> imperative that contributors use time effectively, and doing this
> makes it a lot easier. And I think it's really important to ingrain it
> in your developers and users to be cleanly about this. I'd really like
> to search by metadata to find bugs/issues relevant to me, but that's a
> bit hard sometimes.
>
> So there are two things I'd like to note here, to try and alleviate this:
>
> 1) There needs to be a CONTRIBUTING.md file in the Nixpkgs. The
> reason for this is that we want it to pop up when you file bugs or
> pull requests. For example:
> https://github.com/thoughtbot/factory_girl_rails/issues/new
>
> There is a very clear warning at the top to read about how to
> contribute. This can have a quick note that says things like A)
> "Please label any bugs you file as you see fit" B) "Format your
> messages properly" C) "Brush your teeth before going to bed".
>
> This doesn't guarantee people follow protocol, but it will
> hopefully remind people to file bugs following specific guidelines.
>
> 2) The current issue list is really, really confusing and
> haphazard. For example, there is a 'python' label but no 'perl' label,
> despite the fact I just labeled 5 perl PRs! And there's no 'systemd'
> label, or any label about, say, Xorg, the NixOS installer, etc etc. As
> well as no uniformity to color, either :) Also, labels for things like
> 'high' vs 'low' priority don't exist either (really, these should be
> first class concepts)
>
> So, I'd recommend we take a play from the Rust people and handle
> GitHub's simpllistic tracker, and categorize all labels by a
> 'callsign' - a distinctive single letter - followed by the label name.
> For example, a reinvisioning of the current label list with some
> additions might look like:
>
> "Blocker" issues/bugs:
> B-low -- Low priority
> B-normal -- (Default) normal priority
> B-high -- High priority
> B-blocker -- Blocking bug
>
> "Infrastructure" issues/bugs:
> I-security -- Security updates
> I-documentation -- Documentation
> I-mass-rebuild -- Massive rebuild
>
> "Language" issues/bugs:
> L-python -- Python packages
> L-perl -- Perl packages
> L-php -- PHP packages
> L-haskell -- Haskell packages
> L-ruby -- Ruby packages
>
> "Miscellaneous" issues/bugs:
> M-rfc -- RFC (AKA 'feedback-requested')
> M-wip -- Work-in-progress
> M-question -- Questions
> M-meta -- Meta bugs for other issues.
>
> "NixOS" issues/bugs:
> N-installer -- NixOS installer
> N-systemd -- systemd (NixOS only)
> N-xorg -- Xorg/MESA/etc
> N-gnome -- GNOME packaging
> N-KDE -- KDE packaging
> N-kernel -- Kernel packages
>
> "Package" issues/bugs:
> P-enhancement -- Package enhancement, amendment, etc
> P-update -- Update version package
> P-new -- New package
>
> "System" issues/bugs:
> S-linux -- (Default) Linux bugs
> S-darwin -- Darwin bugs
> S-other -- Other platforms
>
> "Invalid/duplicate/wontfix" bugs
> Z-einvalid -- Invalid issue/bug; -EINVAL
> Z-enotabug -- Not a bug; -ENOTABUG
> Z-ewontfix -- Nope; -EWONTFIX
> Z-duplicate -- Duplicate issue
>
> Note that any issue may combine any of these together. It's also a
> much richer set of labels that express more constraints on a bug. A
> few things:
>
> A) There is a specific 'M-meta' to describe metabugs. This should
> have no other labels associated, and should simply list other bugs in
> a checkbox-list specifying what's blocking them.
>
> B) Labels are color-coated by *category*, not individually. This
> means that all 'P' labels are blue, all Z labels are grey, etc. This
> makes things visually clearer as to what they represent, and saves us
> recycling colors all the time.
>
> -----------------------------------
>
> Finally, there are some other associated things I was thinking of:
>
> 1) It would also be nice to adopt an official policy of maintainers
> for the source tree; e.g. who is responsible for certain NixOS or
> Nixpkgs subsystems, or Darwin/FreeBSD support, etc. This makes it
> easier to track down the right person, IMO.
>
> 2) It would be nice if we could do a round-robin scheduling or
> something of maintainers who have to triage issues, even very very
> briefly. Every day one person is elected to just add labels to things,
> and set the ticket to a milestone. Some projects have done this in the
> past; in practice, I think if we had a random selection of people who
> could once a day add labels and set them, and we were diligent, most
> days would only see 5-10 issues needing labels. Which is not too hard;
> appropriate metadata is pretty easy to infer from just reading an
> issue.
>
> -----------------------------------
>
> There are other things I could think of (e.g. milestoning policies for
> a release), but that's getting a bit ahead of myself. So I'd like to
> get the discussion out there.
>
> FWIW, if people are supportive of this, I'd at least be willing to
> author a CONTRIBUTING.md file to the tree, and re-name all the
> existing labels, and add others as specified above (pending a final
> list some of us can come to an agreement on). That can be done
> relatively easily with an overall bet benefit, I think.
>
> --
> Regards,
> Austin - PGP: 4096R/0x91384671
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
More information about the nix-dev
mailing list