# Description of development tools

## Package `blends-dev`

If metapackages are built using the tools inside the `blends-dev` package it can be ensured that the resulting metapackages will work nicely with the same version of `blends-common` package. The goal is to keep necessary changes for the source of the metapackages of a Debian Pure Blend as low as possible when the version of the `blends` source package changes. Thus it is strongly recommended to use the tools described below.

The usage of the tools in the `blends-dev` package might introduce a versioned dependency in the `<blend>`-`config` package from which all other metapackages of the `Blend` in question will depend. This `<blend>`-`config` package instantiates the `Blend` in the common registry for all Blends in `/etc/blends`.

The version `0.7.0` of `blends-dev` uses UDD to generate Blends' metapackages. Currently all Blends' info is stored into UDD. Information such as VCs, description, homepage etc for a Blend can be found into the blends_metadata UDD table. All the info about Blends' tasks and their package dependencies are also stored into the blends_tasks and blends_dependencies_alternatives tables. Having the latter in combination with other UDD tables (such as a table with info about all Debian available packages) provides the ability to check whether a package exists for an architecture or not thus `blends-dev` can generate architecture dependent metapackages.

The best idea to use the tools provided by the `blends-dev` is to put a `Makefile` into the build directory containing one single line

       include /usr/share/blends-dev/Makefile

(see `/usr/share/doc/blends-dev/examples/Makefile`). Users using `blends-dev 0.7.0` on existing Blends which have more than one releases might encounter some `Makefile` errors for more info see statusdump and changelogentry. Using this `Makefile` all tools that were contained in `blends-dev` package versions before 0.4. These tools are moved to `/usr/share/blends-dev/` because there is no need to call them directly. Here is a list of the `make` targets.

### Blend`-tasks.desc`

This target generates a `task-description.template` file. The template can be converted to a proper description file that is used in `tasksel` to enable selecting the tasks belonging to the Blend. The initial template contains all the needed package dependencies for a Blend. But because some packages might not be available for a(or multiple) architectures the template uses the following syntax when specifying packages:

     package1 [!arch1 arch2]

That says do not include the package1 in the taskdescription file when arch1 or arch2 is used. When a Blends' `orig.tar.gz` is generated, the initial template gets converted from the `blends-dev rules` file to a proper taskdescription file. The conversion is filtering out the packages which are not available for the host's (where the `orig.tar.gz` is generated) architecture. This make sure that the taskdescription file will not include package which are not available for the target architecture. Finally the file will be moved to the `blend``-tasks`. All information about Blends package dependencies is obtained from the UDD.

### `debian/control`

The `debian/control` file of a Blend metapackage source archive is auto generated out of dependencies that are specified in so called `tasks` files. The rationale behind this is to enhance flexibility about changes inside the Debian package pool where new packages might appear and others might be renamed. The `tasks` just define which dependencies the Blend maintainer group wants to be fulfilled and the script `blend-gen-control` using UDD verifies whether these dependencies exist in a specified package pool. A package pool can be considered as the packages available for a combination of distribution, component and release values. By default when creating metapackages debian,main,testing values are used to "create" a package pool from UDD. Once a Blends' dependencies are verified the `debian/control` file is generated according to the available packages. This works not only for the Debian package pool containing the distributions stable, testing and unstable, but also when building your metapackages against a different package pool of a Debian based distribution. This is for instance used to create the SkoleLinux metapackages.

As mentioned in the previous section, using UDD in Blends' tools provides the ability to generate architecture dependent metapackages. Thus the generated `debian/control` specifies for every task source target as architecture value:

    Architecture: any

Specifying `any` indicates that the source package isn't dependent on any particular architecture and should compile fine on any one. To fulfil this in case of missing packages `control` file uses the following syntax:

    Depends: package1 [!arch1 !arch2]

If a package is not available for a specific arch, exclude it from it. So the above example says: depend on package1 but not when architecture arch1 or arch2 is used. More info about `debian/control` syntax can be found in Debian Policy Manual

The syntax of the `tasks` files which serve as the central database for the information in the `debian/control` file is defined by RFC822. Some of the tags were mentioned formerly in packageslist to explain how the `tasks` files can be used to create the web sentinel pages. In order to write valid task files it is mandatory to separate a task file paragraph by an empty line. Otherwise the task pages of the web sentinel will not be built correctly. Now the tags that influence the creation of the `debian/control` file are given.

Depends

- Will be turned to "Recommends" in the resulting `debian/control` file. The rationale behind this is to enable installing the metapackage even if a package belonging to this task is missing for whatever reason. It also allows finally to remove the metapackage. This makes even more sense since `apt-get` considers "Recommends" as default installation targets.

Recommends

- The packages that are listed as "Recommends" in the tasks file should be installed on the machine where the metapackage is installed and which are needed to work on a specific task.

Suggests

- Use "Suggests" for packages of lesser importance that might be possibly useful, or non-free packages.

  If a package is not available in the package pool of the target distribution when creating the `debian/control` file inside the meta package source archive any "Depends" or "Recommends" are turned into "Suggests" to enable a flawless installation of the metapackage. Generally packages are concerned as missing if they do not exist into Debian main component(default is testing release). Packages that are specified with "Suggests" will not be taken over to the `tasksel` control file (Blend`-tasks.desc`, see blends-tasks.desc) but only to the list of suggested packages of the according metapackage.

Ignore

- The "Ignore" key can be used as kind of "Soft-Suggests" to put a package on the radar of the Blend. Packages that are tagged with Ignore will not be taken over into the list of dependencies inside the `debian/control` file of the resulting metapackage neither to the Blend`-tasks.desc` control file for `tasksel` but will be taken over onto the installation medium of a Blend in case there is some space left. This key becomes especially important for specifying not yet packaged software that might be packaged in the future (prospective packages). This is explained in detail in the paragraph about the web sentinel (see packageslist).

Avoids

- The "Avoids" key specifies existing packages that will be listed in the the `debian/control` file as "Recommends" or "Suggests" but, should not go to a installation medium (CD, DVD, etc.) that might be produced by the Blend. A reason to avoid a package might be that it belongs to the non-free section.

Example:

    Task: finest
    Description: the finest software

    Depends: foo

    Depends: bar

    Suggests: foobar

### `statusdump`

This target generates a json file containing the latest package dependencies for the selected Blend. It parses the files from the `tasks` directory and generates a `blend``_version.json` into a `dependency_data` directory. As `version` it gets the latest version specified in the Blend's `debian/changelog` file. In case the `dependency_data` directory does not exist into a Blend's root directory it automatically creates it.

A user can also generate a json dependencies file manually using the `tasks_diff` script. The script can be called from a Blend's root directory:

     /usr/share/blends-dev/task_diff --status-dump --tasks .  --output blend_version.json

If the user does not specify the output argument the script by default will generate the json file under the `tasks.json` name in the current directory.

Note: in case a user needs to generate a json file for a previous release (rather than the latest) to get the `changelogentry` (see changelogentry) target to work, must keep the following thing in mind: The user must provide to `task_diff` script the root directory of a previous Blend release (through the --task(-t) argument). He should also save the output into the `dependency_data` directory into the latest Blend release providing manually the name `blend``_version.json` (through the --output(-o) argument:

    /usr/share/blends-dev/task_diff --status-dump -t blend/tags/previous/ -o latest_blend/dependency_data/blend_version.json

For example if the name of the Blend is `myblend` and the release is `0.2.0` then the json file must have the name `myblend_0.2.0.json`

### changelogentry

This target compares the latest and the previous Blend release and dumps the tasks' package differences. It reports the added/removed packages per task (or added/removed task files) between releases. This "report" is automatically added into the `debian/changelog` in the latest release section under the file's manual changes. In case a previous difference report exists, it overrides it. In case a Blend does not have more than release (initial release) then this target is ignored.

In order the comparison to be properly performed the `blend``_version.json` files for the two latest releases must exist under the `dependency_data` directory. In case any of the previous files is missing then the target will fail with an error (specifying the missing version_file). The json file for the latest release is automatically generated from the `statusdump` target so it this will not cause the problem.

This changelog entry is a new feature so the problem of this target failing (because of a missing json file) will appear for existing Blends which have more than one releases and do not have a `blend``_version.json` for the previous release under their `dependency_data` directory. Usually Blend's releases are tagged into the VCs, so the previous problem can be solved by generating the dependency json file for the previous release (using a previous VCs tag). This can be done by calling manually the `task_diff` script (see statusdump)

### Apt `sources.list` files in `/etc/blends/`

These files are used by

blend-gen-control

1 to build valid `debian/control` files that contain only available packages in their dependencies. This enables building meta packages for `stable`, `testing`, `unstable` or even a completely different distribution that has valid `sources.list` entries. The file `/etc/blends/control.list` is used as default for

blend-gen-control

1 and usually is a symbolic link (see

ln

1) to `sources.list.``distribution`. It might be changed using the `-s` `dist` option of

blend-gen-control

1.

TODO: Either parse the available `/etc/apt/sources.list` or use a sane debconf question to use the "nearest" mirror.

### Templates in `/usr/share/blends/templates`

The directory `/usr/share/blends/templates` contains templates that can be used to build a `<blend>``-config`, which uses the tools that are contained in the `blends-common` package, and are useful to manage `<blend>` user groups (see userroles).

## Package `blends-common`

This package creates a common registry for all Blends in `/etc/blends`. Each Blend should put the files that are used into a subdirectory named like the Blend of `/etc/blends`. The `blends-common` package installs a common configuration file `/etc/blends/blends.conf`, which can be used to influence the behaviour of the tools described below.

###

blend-role

8

NAME

- `blend-role` - add/remove roles in registered Debian Pure Blend

SYNOPSIS

- `blend-role` `add|del` `Blend` [`Role`]

DESCRIPTION

- Add/remove (register/unregister) `Role` for the specified `Blend`. If `Role` is not specified, it's assumed to be named like `Blend`.

OPTIONS

- `Blend`

- A registered Blend in /etc/blends, for example one of `med`, `junior`, `edu` or `science`

AUTHOR

- Andreas Tille tille@debian.org, Cosimo Alfarano kalfa@debian.org.

###

blend-update-menus

8

NAME

- `blend-update-menus` - add menu of metapackage to all Blend users

SYNOPSIS

- `blend-update-menus` [`--blend Blend` | `--user user`]

DESCRIPTION

- blend-update-menus behaves differently depending on who run the command:

  If it is called by a user, it adds, and keeps updated, menu entries for the user who runs it.

  If it is called by root, it adds and keeps updated user's menu entries (see menu package for users' menus) for all users who belong to the group of the specified Blend, or only for a specified user, depending on which parameter is passed to the script.

OPTIONS

- `Blend`

- one of the installed Blends, listed in /etc/blends/, for example (if installed: `med`, `junior`, `edu` or `science`.

`user`

- system user

AUTHOR

- Andreas Tille tille@debian.org, Cosimo Alfarano kalfa@debian.org.

###

blend-user

8

NAME

- `blend-user` - add/remove user to Role of a registered Blend

SYNOPSIS

- `blend-user` `add|del` `Blend` `user` [`Role`]

DESCRIPTION

- Add/remove user to a `Role` of the specified `Blend`. If `Role` is not specified, it's assumed to be named like `Blend`

OPTIONS

- `Blend`

- A registered Blend in /etc/blends, for example one of `med`, `junior`, `edu` or `science`.

`user`

- user to add

`Role`

- the role in the `Blend` that `user` will assume

AUTHOR

- Andreas Tille tille@debian.org, Cosimo Alfarano kalfa@debian.org.

###

blends.conf

5

NAME

- `blends.conf` - configuration for Debian Pure Blends registry

DESCRIPTION

- This file is sourced from shell scripts inside the Debian Pure Blends package `blends-common` and thus it has to follow shell syntax. The variables that are set inside this configuration file can be overridden by special Blend configuration files `/etc/blends/``Blend>`/`Blend>``.conf` for each single Blend.

SYNTAX

- The following variables can be set:

  `DBBACKEND`

- Set the backend for the user role management system. Currently the only implemented role management system is `unixgroups` but others might be implemented later. Unsetting this variable leads to use no roles at all.

`UPDATEUSERMENU`

- If this is set to `yes`, the user menus of meta packages can be created automatically at install time of the package if the postinst script of the package allows this. It is suggested to use this option in the specific configuration files of a special Debian Pure Blend that override the settings of the general configuration file.

`SHAREDIR`

- Set the base directory for the user role management system. While this is more or less a feature for debugging this might be also used otherwise.

`DRYRUN`

- This variable can be set for debugging. Normally it should be left unset (NOT set to `false` or anything else!). If set to `true` a dry run of the tools is performed or `echo DRYRUN:` would print debugging information.

`DEBUG`

- If set to `1` debugging mode is switched on.

SEE ALSO

- blend-role

  8,

  blend-update-menus

  8,

  blend-user

  8

AUTHOR

- Andreas Tille tille@debian.org, Cosimo Alfarano kalfa@debian.org.

## How to develop new normal packages in Pure Blends

Sometimes you may want to develop new packages for your Pure Blend. It is almost identical to creating a new debian package. But in Blends framework you can do it more easily.

Assume that you have already had a well-defined blend. That is, those necessary files like changelog, compat, copyright and control.stub are in your debian/ folder, maybe with postinst.stub or prerm.stub for task metapackages. Notice that, all these control and maintainer scripts are just for metapackages defined in your task, not for your new, normal packages since they aren't listed in task.

When you want to develop and add new packages, just like debian packaging, you need to add package information and parameters in the control file. However in blends, you should add them in debian/control.stub instead of defining their own control file. For the maintainer script files, you need to add them in debian/ folder. However, notice that the maintainer script files should not add ".stub" or blends-dev won't process them.

For example, in blend "foo", we want to add a new normal package "bar". In debian/control.stub, we may have:

    Source: foo
    Section: misc
    Priority: extra
    Maintainer: Foo Team <debian-foo@lists.org>
    Uploaders: Foo bar <foo-bar@nowhere.com>
    Build-Depends: debhelper (>= 7), blends-dev (>= 0.6.91)
    Standards-Version: 3.9.3
    Dm-Upload-Allowed: yes
    Homepage: http://foo.bar/

    Package: foo-bar
    Architecture: all
    Description: sample blends and packages

In the control.stub it defines foo blends, and a new package foo-bar. Then, in the debian folder, we can add our own preinst, postinst, prerm, and/or postrm maintainer scripts. However the filename should be foo-bar.preinst, foo-bar.postinst, ... etc., instead of foo-bar.postinst.stub, since foo-bar doesn't exist in tasks.

After those control files are done, you can use

    make dist

to create blends control files, and then use

    debuild

to start packaging.

## Working with the source repository in `Git`

Sometimes it might be interesting for developers to check out the latest code of the Blend tools or a special Blend code for the meta packages. In communication the directory layout of the `Git`-repository was described. How to derive the Debian packages from this layout?

Checkout

- For the Blend tools and this documentation

        gbp clone git@salsa.debian.org:blends-team/blends.git

  or for the Debian Pure Blend `BLEND-name`

        gbp clone git@salsa.debian.org:blends-team//BLEND-name.git

Build source package

- Change into the created directory and type

        make -f debian/rules get-orig-source

  This creates a `tar.gz` source archive of the packages you want to build. For your personal comfort you can create a file `Makefile` in your package source directory containing

        #!/usr/bin/make -f
        include /usr/share/blends-dev/Makefile

          Which enables you to simply say

          make dist

  to create the source tarball.

Build Debian packages

- gbp buildpackage

The current Debian Med packages provide a working example how to use the tools described below.

## How to create tasks and bugs pages of web sentinel

In sentinel the creation of so called tasks pages packageslist and bugs pages bugs was described. These pages are automatically build by a cron job on blends.debian.org from the current state of the tasks files in the Git repositories of all registered Blends.

To know what a valid `<blend-name>` might be have a look into `/srv/blends.debian.org/webtools/webconf`. Each Blend has an according config file there. Leave out the `.conf` extension and you have a valid `<blend-name>`.

In case you are planing some more experimental changes there is another host (sponsored by rackspace.com) called `blends.debian.net` which is running a copy of UDD and also hosts the latest development snapshot of the Blends web tools. Just ask Andreas Tille tille@debian.org in case you like a login on this host.

The code which builds web and tasks pages is available in Git at `https://salsa.debian.org/blends-team/website`. It is using the Genshi templating system which enables influencing the layout of the pages by editing the templates in the `templates` directory. You can also influence some parameters of the web pages in the configuration files in the `webconf` directory. Last but not least you can provide translations for the web pages in the `po` directory.

Please note that the `css` and `js` files which are influencing the layout of the automatically created pages are in the same area as the static web pages (see below staticwebpages).

## Editing static web pages of Blends on blends.debian.org

A very simple entry page is created for each Blend which is linked from the list of all Blends. Most probably the maintainers of a Blend want to enhance this page a bit. Maintainers of a Blend should care for this index page which currently is just featuring links to the automatically updated pages as well as an image which shows the activity of the relevant mailing list. Maintaining these static pages is not a "service" which is done for you. The maintainers of a Blend should care for this!

The static pages are maintained in Git at `https://salsa.debian.org/blends-team/website` in the `websites` directory. A cron job is watching changes on Salsa and will update the static web pages.

## Description how Blends relevant data are gathered and stored

All data relevant for Blends are available in Ultimate Debian Database (UDD). Here is a description of the data gatherers that were explicitly invented to provide information for usage in Blends websentinel.

### Packages in Debian ftp new queue

New packages might reside for some time in the so called new queue until a member of the ftpmaster team has evaluated the package as fit for the Debian distribution. To enable informing our users about that status the new queue is parsed and the information displayed in the web sentinel pages. The actual gatherer code can be found in UDD Git and is split into a Script that fetches the data and a Parser which injects the data into UDD.

### Machine readable data in Git repositories of Blends and some packaging teams

Several machine readable data of packages are parsed from Git repositories. There are data that might be more up to date in Git (for instance scientific publication data or some registry entries). There are also data for not yet available packages a Blends team is working on. To enable referring to this data some selected Git repositories are parsed. The parser for Salsa is available in the website Git of the Blends project. It is running in a daily cron job on the host blends.debian.net and creates an archive with all the machine readable files found in the specified Git repositories.

This archive is read into UDD by an UDD importer script in a cron job which is totally independent from the job that collects the data from Salsa. Due to the disconnected jobs running on different hosts there might be some undetermined delay between changing some metadata in a Git repository and the time when the data are available in UDD.