OpenOffice Child Workspace Tools

Author: Jens-Heiner Rechtien

Last change: 2004/08/12

Last change by: Jens-Heiner Rechtien

Version 0.91

Please note that this document will be updated from time to time. Find the latest version here: http://tools.openoffice.org/dev-docs/ooo-cws-tools-doc.sxw

Table of Contents

1 Introduction	2

2 Naming Conventions	2

3 EIS: the Web Interface	3

4 Prerequisites	4

4.1 Required Perl Modules	4

4.2 Config File $HOME/.cwsrc	4

5 Child workspace creation	5

6 Working with child workspaces	6

6.1 Tune to the CWS environment	6

6.2 Add tasks to the CWS	6

6.3 Add modules to the CWS	6

6.4 Commit and analyze the changes	6

6.5 Resynchronize and integrate the CWS into the MWS	7

7 Reference	9

7.1 cwscreate	9

7.2 cwsadd	9

7.3 cwsaddtask	9

7.4 cwsanalyze	10

7.5 cwsintegrate	10

7.6 cwsresync	11

7.7 cwsquery	12

7.8 cvs commit	12

Introduction

The idea behind child workspaces is simple: Isolate changes until they are finished and thoroughly tested by the Quality Assurance (QA) engineers. The method of choice is also simple: Use a standard source code control mechanism – a branch – to separate the not- yet-ready changes from rest. The OpenOffice.org source code control system CVS fortunately offers very powerful possibilities to create, modify and merge branches. These possibilities alone are sufficient to sustain a child workspace development model. But then, a little more comfort never hurts. This paper describes a set of tools which encapsulates most aspects of child workspaces. These tools do not only make work with child workspaces more easy, they also impose a set of conventions on the usage of child workspaces. Additionally, they allow analysis of the source code changes done in a child workspace and a complete tracking of changes from a high-level feature-oriented view down to single revisions of files.

Naming Conventions

The following shortcuts and notations are used throughout this document:

CVS, Concurrent Versioning System: Our source code control system.

CWS, Child Workspace: A child workspace is a workspace which has been split off a master workspace at a certain milestone. A child workspace is owned by the child workspace owner and has a physical representation on the disk. A CWS is represented by a branch in CVS. A freshly created CWS does not yet have any modules associated, but modules are added during the lifetime of the CWS.

MWS. Master Workspace: A master workspace represents the road to a product. It is roughly equivalent to the old workspaces like 'SRX644'. Builds created based on the MWS represent the milestones to the product. Only tested and approved CWSs are integrated into a MWS. A MWS is represented by a branch in CVS. A MWS should always be in a releasable state.

Milestone: A milestone is a MWS build on the way to a product. Milestones are equivalent to 'minors' in the old development model. Milestones are named with a sequence of letters and digits starting with a letter. Example: m45.

EIS, Environment Information System: A database driven information system where CWS related (and other) data is stored. Provides a web interface for visualization.

Module: A single CVS module. Some people call them 'projects'. Please use the term 'module'.

EIS: the Web Interface

The web interface (http://eis.services.openoffice.org/EIS2/servlet/GuestLogon) provides a convenient way to browse all the information about the existing CWSs. It offers several views on the CWSs, sorted either by master workspaces, milestones or releases. It's possible to search for a CWS and to view overall statistics. A click on the CWS name will lead to a detailed overview of the selected CWS. The status is displayed, the members of development and QA working on it and which tasks have been assigned to the CWS. Note that the task are linked to IssueZilla for easy lookup. After integration of a CWS there is also a detailed list of changed files with revisions, task-ids and authors. Another important piece of information are the 'creation milestone', 'current milestone' and the 'integration milestone' entries. Additional fields for comments complete the view on the CWS.

Most entries are self-explanatory, but the possible states might deserve a closer look. They have a precise defined meaning.

planned: A CWS with this state is planned, but not yet physically existent. Thus no code has been changed, it's not even yet decided from which milestone the CWS will be created. Having this state available is useful for long term planning, resource acquisition etc.

new: CWSs with this state have been created, they do have a physical representation somewhere. All development is done while the CWS is in this state.

ready for QA: The developers think they are ready. They have prepared installation sets and submitted them to QA. If QA accepts the CWS the state is usually advanced to nominated. If they find bugs in the new stuff or even regressions they will set the state back to new.

approved by QA: A special intermediate state used only for bug fix releases which need an even more controlled approach. QA approves a CWS but program management has the final say if and when something goes into the master.

nominated: Set by QA after accepting a CWS. This is the point where release engineering will take the CWS and integrate it into the master.

integrated: Set by release engineering after the integration. A CWS in this state is considered done.

canceled: A canceled CWS has been abandoned, no more work will be done on it.

The rest of this document is a technical description on how to work with the CWS tooling. Someone with only a casual interest in OpenOffice development might want to stop reading here. Most of the tools require OpenOffice 'commit rights'.

Prerequisites

Required Perl Modules

The CWS tools are mostly perl scripts. These scripts use several perl modules which are not commonly available in a perl installation:

XML::Parser	expat based parser for the new XML based build lists.

Crypt::SSLeay	for SSL encrypted SOAP connections.

SOAP::Lite	access the SOAP based CWS web service.

All modules are available via CPAN, install them with:

% perl -MCPAN -e 'install <module>'

Config File $HOME/.cwsrc

The CWS tools rely on some configuration entries in $HOME/.cwsrc.

Example:

[CWS_CONFIG]

# Network proxy. Comment in if you need to access the net via a proxy.

# Example: PROXY=http://myproxy.company.com:8080

# PROXY=

# CWS Database server (SOAP). It's possible specify to several backup server,

# currently there are none.

CWS_DB_SERVER_1=https://eis.services.openoffice.org/soap/servlet/rpcrouter

# Your CVS login (for authentication with the CWS database).

CVS_ID=

# Your scrambled CVS password (for authentication with the CWS database).

# Take this one from .cvsrc

CVS_PASSWORD=

# OOo CVS tunnel

# Example: CVS_SERVER_ROOT=:pserver:cvs_id@mytunnel.mydomain:/cvs

CVS_SERVER_ROOT=

# Local OOo CVSUP mirror (optional)

# Example: CVS_MIRROR=:pserver:cvs_id@mycvsup.mydomain:/cvs

#CVS_MIRROR_ROOT=

# Path to the cvs binary (optional)

# Example: CVS_BINARY=/usr/bin/cvs

#CVS_BINARY=

Child workspace creation

Before a a child workspace can be created it's necessary to checkout OpenOffice – preferably the milestone from which you want to create your CWS – and run configure.

Example: Create a CWS with the name 'foo1' . It's based on the milestone 'm45' of the MWS 'SRC680'.

%
% cvs -d <cvsroot> co -rSRC680_m45 OpenOffice
% cd config_office
% ./configure
% source <the configured script>

Now it's possible to create the CWS. Execute the cwscreate command.

% cwscreate SRC680 m45 foo1

This command first checks if a connection to the EIS web service is available. It then updates the OpenOffice modules to the requested milestone and finally registers the freshly created CWS with EIS.

It's not strictly necessary to have a complete OpenOffice workspace around, because the '-a' switch of cwscreate can check out the workspace for you, but at least modules 'config_office' and 'solenv' have to be around for the CWS tools to work .

Working with child workspaces

Tune to the CWS environment

The environment variable CWS_WORK_STAMP is of special importance for the CWS tools; it must be set to the name of your CWS. All CWS tools will refer to this environment variable to find out on which CWS we are working.

Example: Use the CWS created above:

% setenv CWS_WORK_STAMP foo1

Add tasks to the CWS

Now let's assume we want to implement a feature with the task ID '#i4711#' and fix the related bugs '#i42#' and '#i666#' together in this CWS. The changes are distributed over three modules, let's say 'sfx2', 'framework' and 'desktop'.

Adding tasks IDs to a CWS is done with the cwsaddtask tool:

% cwsaddtask i4711 i42 i666

Fire up a browser and view the EIS entry for the CWS. Note that the task IDs appear prominently in the web front end. Please assign a QA engineer to be responsible for that particular CWS. Keep the task ID list current through the lifetime of the CWS. The responsible QA engineer will test the changes according to the specifications, test plans and bug descriptions of the task IDs which are registered in the EIS.

Add modules to the CWS

It's time to do some coding. Add the needed modules with the cwsadd tool to the CWS:

% cwsadd sfx2 framework desktop

The cwsadd command creates the CWS branch in these modules, updates the modules to the branch label, tags them with a further administrative tag called the anchor tag and finally registers the modules with the EIS.

All files in the CWS instance of the modules should now carry the CWS branch label as a sticky tag. The CWS branch label is of the form cws_<mws-name>_<cws-name> , in our example it is cws_SRC680_foo1.

Commit and analyze the changes

Committing the changes is easy. Just use the cvs commit command. The sticky tag ensures that the changes go the CWS branch. Just don't forget to specify the task ID in the revision comment.

After we have finished our work we'll need to create an installation set. Change into the 'instsetoo' module and build it. The installation set is then created in the output tree.

We likely want to know which files have changed in our CWS and if they can be integrated without conflicts into the MWS. The cwsanalyze tool determines all changed files in the CWS and does a trial merge with the MWS for a conflict analysis.

% cd /tmp

% cwsanalyze all

The result is a list of all changed files and a notice as to whether they can be merged back into the MWS without conflict. Please use an empty scratch directory for conflict analysis, the trial merge is a destructive operation. The tool leaves the result of the trial merge in the scratch directory, so that it's possible to examine the result of the merge operations.

Resynchronize and integrate the CWS into the MWS

OK, we have done our work. Now the QA has a look at the result. But by now a lot of other CWSs have been integrated into the MWS since we have created our CWS. To be able to judge if our changes still play well with the current state of affairs we should resynchronize our changes with the master.

Let's say we created our CWS based on the milestone 'm45', but the latest milestone is 'm50'. The cwsresync command can resynchronize single files directly in your CWS copy. But, if we plan to resynchronize a whole child workspace, it's far safer to do the CVS operations in a scratch directory. Here the scratch directory is /tmp.

% cd /tmp

% cwsresync -m m50 all

... solve conflicts ...

% cwsresync -c all

% cd <workspace>

... update files in your CWS copy ...

% cwsresync -l m50

% cwsresync -r (optional)

% cd <workspace>/config_office

% ./configure

% source <configured script>

The first command merges the changes from minor 'm45' to minor 'm50' on the MWS into the CWS. This might lead to conflicts, we need to solve them before we commit the merges onto the CWS branch. The merge mode of cwsresync leaves files with the extension <file>.resync in the scratch directory. These files are used to communicate with the commit mode of cwsresync. You can use them to add an additional comment to the forthcoming resync revision, for example how you solved a conflict. Just add a line at the end of the file. In many cases cwsresync will not really create a new revision on your CWS branch but just move some tags. In this case it's not possible to add a comment.

If every conflict is solved then we commit all changes with the cwsresync -c all command.

Next we update our modules and the current milestone information with the cwsresync -l command. Finally we can remove the module output trees with cwsresync -r, it's likely that the content is incompatible and thus better be removed.

For all purposes our CWS is now based on the new milestone m50. Just don't forget that we have to redo the configure command to reflect any changes in the configuration.

The cwsresync command is smart enough to avoid the dreaded 'repeated-merge-syndrome'. This is done by keeping an administrative tag called 'anchor tag'. The anchor tag for our example is CWS_SRC680_FOO1_ANCHOR.

Now we can rebuild our stuff and hand it over to the responsible QA engineer. If everything is okay he/she will approve the CWS and nominate it for integration. Release engineering then takes the CWS and integrates it with the cwsintegrate all command into the MWS.

All revision information including task IDs and authors are finally transferred to EIS. A CWS has reached the end of its life when it is successfully integrated.

Reference

All tools with the exception of cwscreate require a properly tuned CWS environment:

% source <configured script>

% setenv CWS_WORK_STAMP <cws_name>

For cwscreate sourcing the configured script is good enough.

cwscreate

Synopsis:

cwscreate [-h] <mws_name> <milest.> <cws_name>

Options:

-h help.

-a checks out modules.

Used by: child workspace owner

Example:

cwscreate SRC680 m45 foo1

Creates a new CWS, updates all files in the configured workspace to the specified milestone and finally registers the CWS with the EIS database.

cwsadd

Synopsis:

cwsadd [-h] <module> [module ...]

Options:

-h help.

-a force checkout.

Used by: child workspace owner

Example:

cwsadd sfx2 framework desktop

Adds one or more modules to the child workspace. Tags all files in the module(s) with the child workspace branch tag and the anchor tag. Registers modules with CWS in EIS.

cwsaddtask

Synopsis:

cwsaddtask [-h] <taskid> [taskid ...]

Options:

-h help.

Used by: child workspace owner

Example:

cwsaddtask i4711 i42 i666

Adds one or more task IDs to the child workspace. Registers them with EIS.

cwsanalyze

Synopsis:

cwsanalyze [-h] [-n] [-d dir] <all|module> [module ...]

Options:

-h help.

-n disable conflict check.

-d dir operate in directory dir.

Used by: child workspace owner

Example:

cwsanalyze -d /tmp sfx2 framework

Analyzes the specified module / all CWS modules. Reports all changed files. By default tries to merge files and reports if merge operation was successful or if a conflict occurred. For conflict check, cwsanalyze needs to operate in a scratch directory. For security reasons it will complain if the scratch directory contains CVS modules. If conflict check is disabled (switch -n) it will operate on CVS server alone, thus no scratch directory is necessary.

cwsintegrate

Synopsis:

cwsintegrate [-h] [-d dir] <all|module> [module ...]

Options:

-h help.

-d dir operate in directory dir.

Used by: master workspace owner / integrator

Example:

cwsintegrate -d /tmp all

This is essentially the same tool as cwsanalyze but does additionally commit the changes onto the MWS branch. It needs a scratch directory for operation. Checks if a CWS is nominated. Will refuse to operate in a directory which contains CVS modules. If the tool detects a conflict it won't commit the changes immediately but just writes out a merge comment file. The integrator has to solve the conflict and commit the resolved file manually. He/she should use the following command for this:

cvs commit -F <file>.mergecomment <file>

The tool collects all the revision information and annotates the merge revision with these information. Additionally it transfers revision information to EIS. After successful integration it sets the CWS status to nominated.

cwsresync

Synopsis:

cwsresync [-h] [-d dir] -m <[mws:]milest.> <all|mod.|file> [mod.|file ...]

cwsresync [-h] [-d dir] -c <all|module|file> [module|file ...]

cwsresync [-h] -l <[mws:]milestone>

cwsresync [-h] -r

Options:

-h help.

-m merge changes from MWS into CWS.

-c commit the merged files to CWS.

-l updates modules, register new milestone.

-r remove output trees.

-d dir operate in directory dir.

Used by: child workspace owner

Example:

cwsresync -d /tmp -m SRC680:m50 all

cwsresync -d /tmp -c all

cwsresync -l SRC680:m50

cwsresync -r

This tools resynchronizes the child workspace with a newer milestone of the MWS. It is typically used by the CWS owner shortly before handing the workspace over to the integrator, or is used to prevent version skew on a very long lasting CWS. Due to the complexity of the operation, the cwsresync operation comes in three operation modes. If invoked with the -m switch it will merge the changes between the last resynchronize operation (or the creation of the CWS if no resynchronization has been done yet) and the specified milestone into the CWS. The merge operation can be done on all modules of a CWS, single modules and single CVS files. It operates either in the directory which is specified by the -d switch or in the current directory. If invoked outside of a module, it will check out the module before trying to merge. It will report successful merges, conflicts and tag moves.

The resynchronized files can be committed to the CWS via the commit mode of cwsresync, which is invoked with the -c switch. In this mode, the tool will search for resynchronization files (<file>.resync) created in merge mode and act according to the instructions in these files. It is possible to add a comment to the end of a resynchronization file, if the instruction calls for creating of a new revision on the CWS branch.

Next, using the link mode (invoked with -l) updates the modules which have not been added to the CWS and registers the new milestone with EIS .

Finally, if invoked with the -r switch, cwresync will remove all output trees in the modules.

This tool uses bookkeeping tags to prevent the 'repeated-merge-syndrome'.

cwsquery

Synopsis:

cwsquery [-h] [-m mws_name] [-c cws_name] <modules|taskids>

cwsquery [-h] [-m mws_name] <latest>

Query modes:

modules query the modules added to the child workspace.

taskids query the taskids assiociated with the child workspace.

latest query the latest milestone available for resync.

Options:

-h help.

-m mws_name name of the master workspace.

-c cws_name name of the child workspace.

Used by: all

Example:

cwsquery modules

Used to query miscellaneous information about the CWS. The same (and more) information can be viewed via the EIS web front end.

cvs commit

Synopsis:

cvs commit [-m “#taskid#: ...”] [dir|file ...]

Options:

-m revision comment, should contain task ID.

Used by: all developers working on a CWS.

Example:

cvs commit -m”#4711#: insert new target” makefile.mk

The standard command to commit changes to a CWS. Requires that the sticky tag is correctly set.

Use '%' as wildcard, as the search expression is inserted in a SQL query.

How should a CWS be named? Good question. For now we stick to a convention: A personal CWS should be named by the OpenOffice.org ID followed by a number. Or a good functional name should be chosen if a team implements a set of features.

CWS creation is still a bit clumsy, suggestions are welcome.

Note that branch labels are by convention in all lowercase letters.

Module wide CVS operations depend on a clean CVS tree. The easiest way to ensure that we have a clean CVS tree is to checkout a fresh one, of course. If we've created a mess during the merge operation, we just throw it away. If cwsresync proves to be rock solid, in place resynchronization will be enabled.

This is essentially the cwsanalyze command with a following commit on the MWS.

The name 'link mode' is historic and not appropriate anymore.

OpenOffice Child Workspace Tools	Version 0.91, last change: 2004/08/12

13 of 13

OpenOffice Child Workspace Tools

Documentation for CWS related Tooling

Jens-Heiner Rechtien

2004-07-06T15:12:07

Jens-Heiner Rechtien