Release history

0.4.1 (2024-06-28)

Server

  • Make debusine:test artifact instantiable.

Web UI

  • Introduce a common base layout with a right sidebar.

  • Implement labels for artifacts.

  • Add specialized view for showing build log artifacts.

Worker

  • Run sbuild with --bd-uninstallable-explainer=dose3 and parse its output.

Quality

  • Fix license classifier in setup.cfg.

0.4.0 (2024-06-24)

Server

  • Add API endpoint to retry work requests.

  • Implement retrying workflows.

  • Give the scheduler Celery worker a different node name.

  • Switch to RedisPubSubChannelLayer.

Web UI

  • Add UI to retry work requests.

Worker

  • Add binary-only NMU support to sbuild task.

  • Use arch-test to provide better defaults for system:architectures.

Signing

  • Add a new signing service. This currently supports generating keys (though currently only in software, as opposed to an HSM) and signing UEFI Secure Boot images with them. A few more pieces still need to be assembled before this is useful.

Documentation

  • Document HTTPS setup.

  • Document signing worker.

Quality

  • Remove now-unnecessary autopkgtest schroot creation from integration tests.

  • Add a “playground” system to manage test object creation and to allow discussion of UI prototypes.

  • Use HTTPS in integration tests.

  • Bump timeout for mmdebstrap integration tests.

  • Reorganize test cases for improved type-safety.

  • Fix cleanup order in an integration test which caused failures on slow architectures.

0.3.2 (2024-06-03)

Server

  • Rename some leftovers of “internal” naming for server tasks.

  • Added method to check if a work request can be retried.

  • Fix Architecture: all matching in sbuild workflow.

Web UI

  • Second iteration on collection UI design.

  • Add base template support for django.contrib.messages.

Quality

  • Fix several race conditions and timeouts that caused autopkgtest failures on slow architectures.

0.3.1 (2024-05-28)

Server

  • Namespace collections under workspaces.

  • Refresh worker from database before marking it disconnected, so that we don’t lose changes made using debusine-admin edit_worker_metadata.

  • Add backend capability to retry aborted or failed work requests.

  • sbuild workflow:

    • Fix task data for Architecture: all work requests.

    • Specify the backend in environment lookups.

    • Defer environment resolution.

Web UI

  • Fix typo resulting in HTTP 500 error in collection detail view.

Worker

  • Handle systemd 256 in incus-lxc executor.

  • Handle dangling /etc/resolv.conf symlinks in environments in the unshare executor.

  • Fix mmdebstrap task to specify the architecture of the chroot.

Documentation

  • Fix several errors in the “Getting started with debusine” tutorial.

  • Adjust “The debusine command” reference to refer to self-documenting --help output.

Quality

  • Skip some integration tests for architectures that weren’t in bookworm.

  • Add enums for artifact and collection categories, to guard against typos.

0.3.0 (2024-05-23)

Highlights:

  • The focus of this milestone is on automatic orchestration of building blocks, to allow tasks to be scheduled for all items of a collection. For example, Debusine can now automatically schedule Lintian tasks for all packages in a suite.

  • Added collections and workflows.

  • Added a new lookup syntax, taking advantage of collections.

Server

  • Add infrastructure for collections.

  • Implement debian:environments collection.

  • Implement debian:suite-lintian collection.

  • Add debusine-admin create_collection command.

  • Store tokens only in a hashed form.

  • Implement debian:suite collection.

  • Move the scheduler to a dedicated Celery worker.

  • Generalize work request notifications into event reactions.

  • Implement basic building blocks of workflows.

  • Implement synchronization points.

  • Implement workflow orchestrators.

  • Implement workflow callbacks.

  • Add --default-file-store options to debusine-admin create_workspace and debusine-admin manage_workspace.

  • Restrict creation of non-worker tasks via the API.

  • Add debusine-admin create_file_store command.

  • Implement scheduling priorities.

  • Implement update-collection-with-artifacts event reaction.

  • Implement collection item lookup syntax and semantics.

  • Implement aptmirror server task.

  • Implement updatesuitelintiancollection task to update a debian:suite-lintian collection from debian:suite.

  • Implement debusine:workflow-internal collection.

  • Add debusine-admin create_work_request command.

  • Implement sbuild and update_environments workflows.

  • Add a _system user for use by scripts.

  • Implement expiry of collection items.

  • Add APIs to create workflow templates and workflows.

  • Add debusine-admin create_workflow command.

  • Add debusine-admin delete_workspace command.

  • Implement expiry of work requests.

Web UI

  • Fix ordering of work requests by task name.

  • Improve rendering of multi-line strings in task data.

  • Show workflow information for work requests that are part of workflows.

  • Show task type in work request lists.

  • Improve handling of expired artifacts in autopkgtest/lintian views.

  • Order a work request’s artifacts by ID within each category.

  • Show the user who created a work request in the work request detail view.

  • Show a notice when a work request’s artifacts have expired.

  • Add workspace detail and collection views.

Client

  • Separate YAML input and output more clearly when running debusine create-artifact or debusine create-work-request.

  • Add debusine manage-work-request command to adjust work request priorities.

  • Add debusine create-workflow-template and debusine create-workflow commands.

Worker

  • Add support for passing extra packages to the sbuild task.

  • Exit cleanly on failure to report a completed work request to the server.

  • Restrict mmdebstrap and simplesystemimagebuild tasks to workers that support the requested architecture, as was done for other tasks in 0.2.1.

  • Only consider the autopkgtest task to have succeeded on exit codes 0, 2, and 8.

  • Remove network-related files that mmdebstrap copies from the host.

  • Allow sbuild to produce no .changes file, so that users can examine the log files of failed builds.

  • Improve “Unexpected artifact type” error from the image cache.

  • Rename autopkgtest task’s environment key to extra_environment.

  • Rename environment_id to environment in all tasks, and support the new lookup syntax.

  • Drop insecure sbuild_options from sbuild task.

  • Rename task data fields in autopkgtest, blhc, lintian, piuparts, sbuild, and updatesuitelintiancollection tasks to support the new lookup syntax, removing _id from key names and accepting single or multiple lookups as appropriate.

  • Correctly tag sid tarballs and images as codename=sid.

  • Don’t purge build-dependencies after build in the sbuild task.

Documentation

  • Move unimplemented features to a new “Development blueprints” section.

  • Add design practices.

  • Rework “Where to start” section in “Contribute to debusine”.

  • Clarify parameters to piuparts task.

  • Clarify the role of Incus when installing a debusine instance.

  • Add design for tasks that update collections.

  • Document work request scheduling and associated worker metadata.

  • Add design for workflows.

  • Document image caching and cleanup.

  • Add design for scheduling priorities.

  • Add design for collection item lookups.

  • Add design for sbuild workflow.

  • Add design for update_environments workflow.

  • Add how-to for setting up APT mirroring.

  • Add example script to automate Incus configuration for workers.

  • Document packages required for Incus VMs.

  • Add example script to populate a debusine instance with example data.

  • Document environment requirements for executor backends.

  • Update “Getting started with debusine” tutorial to use workflows and collections.

  • Add more documentation of worker behaviour.

Quality

  • Validate the summary in debian:lintian artifacts.

  • Drop compatibility with Debian bullseye; debusine now requires Python >= 3.11.

  • Enforce pydantic models for WorkRequest.workflow_data and WorkRequest.event_reactions.

  • Use pydantic models for autopkgtest and lintian views.

  • Fix some tests on non-amd64 architectures.

  • Auto-format HTML templates using djlint.

  • Add infrastructure for more semantic testing of HTML output.

0.2.1 (2024-03-07)

Server

  • Add a Celery worker for server-side tasks.

Client

  • Trim down dependencies slightly.

Worker

  • Require KVM access for simplesystemimagebuild task.

  • Change container to instance in Incus templates.

  • Log task completion.

  • Restrict tasks to workers that support the requested architecture.

Documentation

  • Improve home page slightly.

Quality

  • Enforce mypy project-wide, including all Django components.

0.2.0 (2024-02-29)

Highlights:

  • Added artifact file storage system.

  • Debian developers can use Debusine to run various QA tasks against packages they are preparing. Those tasks can be scheduled through the API or through the web UI.

Note that it is not possible to directly migrate a database installed using 0.1.0. Migrations from this release to future releases will be possible.

Server

  • Implement file storage.

  • Implement artifact handling.

  • Implement expiration of artifacts and their files.

  • Run database migrations on debusine-server package upgrade.

  • Add debusine-admin monthly_cleanup command, run from a systemd timer.

  • Link work requests to workspaces.

  • Add debusine-admin create_user, debusine-admin list_users, and debusine-admin manage_user commands.

  • Link tokens to users.

  • Allow email notifications if a work request fails.

  • Depend on python3-daphne.

  • Ensure all local artifacts are JSON-serializable.

  • Add debusine-admin create_workspace, debusine-admin list_workspaces, and debusine-admin manage_workspace commands.

  • Use WorkRequest workspace in artifacts.

  • Add default expiration delay to workspaces.

  • Add API to list work requests.

  • Make sure the Django app’s secret key is never publicly readable.

  • Mark workers as disconnected on debusine-server startup.

  • Use Restart=on-failure rather than Restart=always in debusine-server.service.

  • Add debusine-admin info command to help with setting up deployments.

  • Add daily artifact cleanup timer.

  • Use pydantic models for artifact data.

  • Add remote, read-only file storage backend for external Debian archives.

Web UI

  • Add web UI for work requests and workspaces.

  • Add login/logout support to web UI, allowing access to non-public workspaces.

  • Allow registering/removing user API keys using the web UI.

  • Allow uploading artifacts using the web UI.

  • Refinements to web UI for work requests.

  • Make Django aware of HTTP/HTTPS state of requests.

  • Fix download error with empty artifact file and document mmap usage.

  • Implement integration with Salsa Single Sign-On.

  • Add lintian view.

  • Polish various aspects of the web UI.

  • Add autopkgtest view.

  • Fetch images for tasks directly, not via a tarball.

Client

  • Rename client’s configuration key from debusine to api-url.

  • Add --data option to debusine create-work-request.

  • Rename debusine work-request-status to debusine show-work-request.

  • Add debusine on-work-request-completed to allow running a command when a work request completes.

  • debusine.client: Drop obsolete silent keyword, and stricter prototype tests.

  • Add debusine --debug option to debug HTTP traffic.

  • Implement a package downloader (dget).

  • Implement a paginated listing API client.

  • Add API client method for listing all work requests.

  • Add debusine list-work-requests command.

  • Add debusine import-debian-artifact command.

Worker

  • Modify sbuild task to use artifacts.

  • Add pre-upload consistency checks on sbuild results.

  • Rename worker’s configuration key from debusine-url to api-url.

  • Upload sbuild log files even if the .dsc file did not exist.

  • Add piuparts task.

  • Add lintian task.

  • Add autopkgtest task.

  • Add mmdebstrap task.

  • Avoid trying to add debusine-worker user in postinst if it already exists.

  • Add image caching for executor backends.

  • Add unshare executor.

  • Port the autopkgtest and piuparts tasks to unshare.

  • Use Restart=on-failure rather than Restart=always in debusine-worker.service.

  • Make tasks check whether their tools are installed.

  • Use a lock to protect execution of the work request.

  • Add blhc task.

  • Add simplesystemimagebuild task.

  • Use pydantic models for task data.

  • Log exceptions in task preparation and clean-up.

  • Add Incus executor (for both containers and VMs).

  • Add a qemu executor, currently only for autopkgtest and sbuild tasks.

Documentation

  • Drop the “slug” field and the “repository” type.

  • Document debian:package-build-log artifact in ontology.

  • Document using local.py to change settings.

  • Create an overview document with an elevator-pitch-style introduction.

  • Add initial design for autopkgtest and lintian tasks.

  • Add initial design for system tarball artifacts and debootstrap-like tasks.

  • Add initial design for tasks building system disk images.

  • Update the description of the sbuild task.

  • Restructure the documentation following the Diátaxis principles.

  • Clarify copyright notice, contributor status and list of contributors.

  • Enable the Sphinx copybutton plugin.

  • Add some documentation for the Python client API.

  • Improve the “Getting started with debusine” tutorial.

  • Add documentation for debusine-admin commands.

  • Add “Install your first debusine instance” tutorial.

  • Add initial design for collections.

  • Refine design for workflows.

Quality

  • Harmonize license to be GPL-3+ everywhere.

  • Support pydantic 1 and 2.

  • Apply mypy, pyupgrade, and shellcheck consistently.

  • Sync (Build-)Depends with setup.cfg.

0.1.0 (2022-09-09)

Initial release. Includes a server that can drive many workers over a worker-initiated websocket connection, where the workers use the server’s API to get work requests and provide results. There is an sbuild task that workers can run.