Scratchbox is a cross-compilation toolkit designed to make embedded Linux application development easier. It also
provides a full set of tools to integrate and cross-compile an entire Linux distribution. Especially Debian is
supported, but is been used also on other Linux distributions. Scratchbox provides a sandboxed build environment which
offers a controlled set of tools and utilities needed for cross-compilation. It is an environment in which it is easy
to assure that the intended versions of libraries, headers and other such files are used during the build.
The test plan covers testing the software implemented or modified in the Scratchbox project before a release can be
established. The reader is also useful to review documents published on project pages  and
should also achieve the Scratchbox install manual  document at hand since design and functionality are described there and not duplicated in this
The document comprises firstly the checklist to confirm the release is containing up-to-dated products for aiming
products consistency. And secondly, the actual testing procedure is identified purposes to describe the testing
methodology used in the project without going into details of the tests. The details and test instructions are covered
in Scratchbox install manual 
The delivery package shall be tested with different Linux distributions with latest, suitable third party packages.
When differences or fault behavior on implementation (i.e. bugs) are found, one of the following alternatives should be
If the bug can be fixed inside the project scope it should be added to the buglist and later fixed.
If the problem seems to be in 3rd party software it should be
added to the buglist,
described in the test report and
reported to the maintainers of the broken software if the bug is not a known problem of the software.
Also minor defects that are not considered test failures should be reported as bugs, but with lower priority. Bugs
are classified to the following priority classes based on the severity:
Application stops, crashes, or ceases to function.
Application is severely restrict, but can continue to run, or application has a security related issues.
Specific functionality is missing or unexpected behavior occurs.
Errors that have a minor impact on use of the software, e.g. usability enhancements.
The release test is succeeded if a release works correctly on Debian testing, Debian stable and the latest Fedora
distributions, with PC and ARM glibc c-library choices. This is valid until further notice of a development progress.
After succeeded testing on Debian stable environment, tests can be run also on other environments.
When the product auditing discovers a failure preventing testing, e.g. packet uncompliances, or the dynamic testing
is to be prevented, because the application ceases to function, testing will be suspended and to be continued when
failures are fixed.
The third party, open-source software packages of latest version are to be checked, compared and confirmed before a
release testing. Each package source compared to, is found in Release notes and in CVS directories. The current
versions of available packages, with the naming convention, can be found in Makefiles under the following named
The version of each 3rd party package used in Scratchbox, should be inspected on Debian web pages  and compared to latest corresponding Fedora package. The updated directory for Fedora
distribution packages can be found, for example, in address . The older package version
of these two named sources should be picked to be a valid package for a Scratchbox release. Notice, often package
deprecations, package splits or package removes are involved. Also, these should be inspected and reported.
All new releases developed in the Scratchbox project shall be tested with the following supported Linux distributions:
The latest Fedora
For the developing purposes some releases are tested infrequently also on other Linux distributions to be tested with, listed below.
Red Hat Enterprise Linux 3 WS
Fedora Core 1
Needless to say that different Linux distributions have differences in function. Therefore the Scratchbox project
documentation covers the Scratchbox Distribution Compatibility
testing  document describing testing procedures for each upon named Linux
Auditing of Scratchbox products (software packages, documentation) is particularly important for insuring the
compliance and sanity of the product delivery. Defects or uncompliances should be detected in early phases. Also
improvement suggestions should be possible to report, since Scratchbox is in continuous developing progress.
When making architectural changes in developing that deviates from the documentation, the developer should be able
to make a bug of it into the Bugzilla under the documentation title.
Before a new release is to be tested and then established, the tasks on the following checklist should be signed,
and deviations or absences should be reported promptly and sign as defects. Before evaluating, the personal Scratchbox
CVS repositories should be updated when necessary.
Check the documentation. Inspect that the documentation is
contains up-to-dated names, numbers, tables or contexts etc. Whenever an architectural change or a new implementation
occurs the project documents should be updated accordingly. Check the contents responds to the configuration.
Check and update testplan: when necessary update tests or
package information e.g. and create a new version testplan.
Check the webpage documentation: generic view of documentation
and particularly inspecting of the user documentation, comparing release notes and consulting for developers of new
Check the CVS repository: evaluate files in doc directories
under CVS repository.
Check the CVS repository modules.
Acquire a list of latest software package versions: (when
necessary implement checks of suitable websites etc.)
Compare packages on CVS and on web release notes: the contents
should respond to current configuration.
Check the software packages versions: make sure the older
version of two package resources, Debian testing release and the latest Fedora release, is available.
Report changes: involve information in report about package
updates, splits, removes or other relevant changes.
Evaluate bug fixe and open bugs: compare bugs on previous
version and evaluate risk spots.
If auditing the products doesn't discover the suspension criteria release package testing is to be run. Tests should
be run firstly on Debian stable distribution, if suspension criteria occurs here, no following tests are need to run on
Acquire and install the latest Debian and Fedora platforms and then the Scratchbox delivery. Since a release version
of 0.9.8, the product delivery is divided into the following named package deliveries:
And support currently the following toolchains:
The release should be tested with all toolchains though emphasizing firstly glibc c-library choices with PC and
The compiling should work in three different environments:
i386, PC development environment
QEMU, target environment emulator
real hardware, in this case it may be IPAQ with ARM (when suitable release version is available, it could be tested
also in other hardware)
The guideline is that the ARM should be compiled with sbrsh
and qemu. And the PowerPC should compile with qemu. Refer to  for detailed instructions for
using qemu CPU emulator. The user is able to select either sbrsh or qemu by
setting environment variable SBOX_CPUTRANSPARENCY_METHOD.
Testing should be run on at least three different types of source software:
'Hello World' application delivered with Scratchbox
GTK2 'Hello World' application delivered with Scratchbox
Debian application package delivered with Scratchbox (See Chapter 4.2 in )
Detailed command line procedures for performing compilation and X application launching can be found e.g. in the .
To be able to cross-compile more complex software which uses 'configure' scripts (such as glib, gtk, etc.) the
Scratchbox CPU Transparency feature has to be enabled. This is done after compiling some software for x86 using the
Set-up the CPU transparency according to following steps that instructions are documented in :
Configure Linux handheld for network settings(Chapter 3 ).
Set-up the NFS environment and start the NFS server (the method varies on different Linux platforms)(Chapter 3.1 ).
Configure Sbrsh feature on .sbrsh file(Chapter 3.2 and Appendix A. ).
Install Debian devkit and Debian rootstrap for GUI libraries to use (Chapters 4.1, 4.2 ).
Build and run simple and complex software with Sbrsh on different targets with different toolchains (Chapters 2.3,
After launching programs check for transparency log file for detailed results.
Check that software compiles and runs correctly. If errors, check first the environment, e.g environment variables
prior to reporting errors.
Uninstall the Scratchbox according to documented (Chapter 5.3 ) procedure and make a