Raw Development TODO Feb 2015

mini-buildd Development TODO

(0) MOTM: Feb 2015

Some notes/news about the release planning concerning official Debian:

We are still in jessie freeze; for mini-buildd release planning, this means (basically purely due to practical considerations) the "1.2 dev kick off" will be delayed at least until jessie is actually released.

In the meantime, I decided to push 1.0.x stable updates via to actual Debian channels through Debian experimental during jessie freeze. Thusly one can see the actual state though Debian eyes, pick it up and do as one pleases, without the extra burden of adding/trusting the Hellfield archive.

As soon as jessie is released, 1.0.6 will be released and hit unstable. I am planning to provide regular stable updates for jessie through Debian jessie-backports.

An initial 1.1.x will, then, hopefully soon hit experimental.

(1) 1.0.x Stable

Subject for patching up stable 1.0.x.

Known/possible issues atm:

  • [BUG] When distribution is wrong (like for example squeeze-wheezy-experimental) mini-buildd keeps var/log/wheezy/_failed/... log cruft (forever it seems)

  • [BUG] chroot removal fails when no appr. archive exists

  • [BUG] chroots: Sources used for bootstrapping stick. 'check' may fail when sources eventually become invalid.
    • => Use/update with current source when doing chroot 'check'.
  • [BUG] Uploader: GPG, not on keyserver: Is removed when running prepare (not intuitive...)

  • [BUG] when sbuild fails in setup stage, sbuild idles around (?)

  • [BUG] (?) 'mini-buildd-tool -q' does not suppress server logs like "[hellfield] I: API call 'getsourceslist' by user 'AnonymousUser'"

  • [BUG] Daemon change (for example: email regex): Needs start/stop to become active.

Fixes for the minor issues ("buglets" below) eventually may or may not be subjects for backports (depending of how invasive the patch is).

(2) 1.1.x Development

(2.0) 1.2.0 Release Goals

1.2 will have secure data channels as only big new feature, and else focus on code stabilisation and cleanup:

  • Support secure data channels (HTTPS, SFTP).
  • Code migration to python3.
  • Code cleanup and speed optimizations (python code, algos, calls, reprepro, ...).
  • Improve documentation.

Be sure it does not:

  • Change SQL scheme, i.e., no incompatible changes to django models.
  • Change to the inter-instance API (except for HTTPS/FTPS support if needed).
  • Change to user API's (except for HTTPS/FTPS support if needed).

(2.1) 1.1.x Development Kickstart

This will most likely take place after the jessie Release; it will mark the point we might not be able any more to merge fixes back from stable.

  • Debian packaging
    • deb: +"mini-buildd-tool" package

    • deb: -"mini-buildd-common" package

    • Cleanup /usr/share/doc/examples/:
      • python-mini-buildd vs mini-buildd
      • examples versus actual package data
    • lintian clean

  • Code cleanups
    • Harden package dependencies to jessie/wheezy+bpo versions to remove as much compat code as possible:
      • Remove doc/apidoc.py
      • django < 1.7
      • ...
    • Bulk-migrate code to python3

    • Fixing up pylint:
      • Fix/cleanup in-code exceptions
      • [WISH] pylint: Move all disabled tests from pylintrc to source code

(2.3) Patches

New add-on like features, w/o need to change much of the infrastructure. Not all may be actually picked up for 1.2:

  • [FISH]: feature/better_buildstatus Show build status and lintian status directly, with colors: i386(BL) amd64(BL)

  • [WISH] API(package_status): Status for package+version: building, installed, failed (for tools polling for build results).

  • [WISH] Integrate ssh upload (using mini-buildd user?) (currently in examples only)

  • [WISH] Include "super-portext" example.

  • [WISH] Config option "mirrors"
    • API call 'mirrors'.
    • Bundle with API call getsourceslist (others?).
    • keyring package: sources.list library
  • [WISH] Status: Add remove/migrate/?? to last packages

  • [CHECK] http://tools.ietf.org/html/rfc5147
    • Link to parts of the build log via fragment to text/plain
  • [WISH] Make passive ftp port range configurable.

  • [WISH] keyring: Auto-migrate keyring packages.

  • [WISH] Library of predefined sbuild --chroot-setup-command snippets (eatmydata, ccache)

  • [WISH] rm libeatmydata as default (?).

  • [WISH] Web API should work with POST; we should use POST by default.

  • [WISH] admin/model objects: Show "used by count" (so we can see what objects may be obsolete).

  • [WISH] Archive: Rethink Archive::mbd_ping (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=740576)

  • API: getsourceslist:
    • 'mini-buildd-tool vm154 getsourceslist --suite='(stable|stable-rollback0)' wheezy' should work
    • 'mini-buildd-tool vm154 getsourceslist --mirror=XXX
  • [WISH] BTRFS chroot backend

  • 'Complete' per-upload config (More ways to influence build via changes magic lines):
    • New flags brainstorm:
      • OK: INTERNAL_APT_PRIORITY
      • DISTRIBUTION_APT_PRIORITY... ?
      • DEB_BUILDOPTIONS
  • [FISH] API: port/portext: --extra-changelog-entry

(3) Further Development Brainstorm

  • SQL schema migration (with django 1.7)

  • PYTHON API for mini-buildd API calls.

  • piuparts check functionality (sbuild?)

  • Sources/Archives: Support partial arch archives (?)

  • Better 'user role concept', better keyring management workflow:
    • Upload-Admin:
      • Liste der Uploader, die auf eine Upload-Queue zugreifen koennen

      • Uploader-Requests ("Anklopfen"):
  • notify: Admin notify on deactivated daemon.
    • ->Daemon.notify for admin messages, repo.notify for packages.
    • ->Daemon.notify for email, rest as before