Native Installer Packaging Process

OpenOffice.org 2.0 / StarOffice 8

Document - ID	Specification Owner	Last Change	Status
	Ingo Schmidt	11.04.03	Draft
Conforms to	
Applies to	Installation
Task ID(s)	i25325
Category	Feature

Abstract

In CWS “newscpzip2” the new packaging process for native installer is introduced. In the middle of this process is the perl program “make_installer.pl” (name can be exchanged any time), which substitutes the existing “scpzip” that is part of the setup2 project. After the creation of new scp modules scp2 and scp2so, the new parser pre2par.pl and the new linker par2script.pl, the make_installer.pl is the last step from scp to native installer. In parallel installation sets using our old setup application can be build using the old scpzip.

Installation sets built with “make_installer.pl” will no longer use our old setup application, but will be RPMs, Solaris-packages or Windows Installer installation sets. The only installation set with a Graphical User Interface (GUI) is the Windows Installer. This is documented in the specification “WindowsInstallerGUISpec.sxw” in parallel to this specification. Another specification for this CWS describes the loader for the Windows Installer “Windows Installer Loader specification“. A further documentation describing the GUI installer for the Unix platforms, which shall be the CNP SDK, is described in another specification “CNP SDK GUI Installer”. But this shall be introduced in another CWS.

i-Team Members

	Name	E-mail Address
Specification Owner	Ingo Schmidt (IS)	
User Experience		
Development	Ingo Schmidt (IS), Oliver Braun (OBR), Dirk Völzke (DV), Christof Pintaske (CP)	
Quality Assurance	Olaf Felka (OF)	
Documentation		

Approved for Implementation

	Approved by	Date
User Experience		
Development		
Quality Assurance		
Documentation		
String Review		

Document Change History

Rev. Level	Change	Initials	Date
1.0	Initial Release	IS	11th March 04

Contents

Glossary	1

1	Motivation	1

2	User Scenarios	1

3	Goals	2

4	Requirements and Dependencies	2

4.1	Requirements	2

4.2	Technical Dependencies	2

5	Competitive Analyses	3

6	Detailed Specification	3

6.1	Parameter of “make_installer.pl” (overview):	3

6.2	Detailed parameter explanation:	4

6.3	The package list file format (non-Windows only)	6

6.4	What makes “make_installer.pl”?	6

6.5	Technical aspects of the Perl packaging process	8

6.6	Error Conditions	10

7	ESP Package Manager (EPM)	10

8	Future Tasks	10

9	Legal Issues	11

10	Notes	11

11	References	11

Glossary

Term	Description
RPM	RedHat Package Manager
PKG	Solaris software package format
MSI	Microsoft Installer, old name for Windows Installer
CNP	Customer Network Product
SDK	Software Development Kit
GUI	Graphical User Interface
ESP	Easy Software Products
EPM	ESP Package Manager

Motivation

In StarOffice 7 the installer was an application of the StarOffice product. This installer application was the same on all supported operating systems. The design goal was to be consistent across platforms. Starting with StarOffice 8 we are more and more focussing on the demands and requirement of larger enterprises. To ease installation and maintenance of StarOffice 8 it will be more important to have consistency in installation for all products on one platform to fit into deployment tools the customer already uses. Therefore we will support the respective installer of different operating systems that is the most accepted or standard. StarOffice 8 is going to support:

Redhat Package Manager (RPM) for Linux

Solaris Packages for the Solaris operating systems

Windows Installer for the Windows OS

Usage of the native installer makes it possible to use the support provided by the operating system. This includes the usage of the operating system specific databases, in which all installed products are registered, and services, which make the installation of different products consistent. The advantage for the user is the consistent look and feel for all installations on his platform and the ease of maintenance for the different software products.

The creation of the native installation sets requires a completely new tooling the the packaging process. In the centre of this new tooling is the perl program “make_installer.pl”, which can generate Windows Installer Installation sets for the Windows OS, RPMs for Linux, Solaris-Packages for the Solaris OS and additionally all packags that are supported by the open source tool “epm”. At the moment this are Debian packages, Mac OS X and many more.

User Scenarios

The new tooling has no GUI and is only used in the internal packaging process for OpenOffice.org, StarOffice and all other products. Required to run the program is Perl, also a zip and an unzip program. For Windows also the Windows installer SDK, which is needed for the creation of the Windows Installer database. For all non-Windows platforms EPM is used. For our inhouse products we need to use a patched EPM, to make the created packages relocatable and to create them in another directory than the default (Linux: /usr/src/RPMS/i586/... , Solaris: /var/spool/packages). Furthermore some environment variables have to be set, to create the pathes in which the source files can be found.

The existence of all needed files and environment variables is checked at the beginning of “make_installer.pl”.

For a detailed presentation of the new packaging process see item 6 (“Detailed Specification”)

Goals

The goal is a tooling which creates native installation sets directly from setup scripts that are created in the scp projects (scp2 and scp2so). At the beginning the new installation sets shall be created in parallel to the installation sets using our own setup application. If the quality of the new native installation sets is sufficient, it is no longer necessary to create installation sets with the old “scpzip”.

Requirements and Dependencies

Requirements

Perl (to execute “make_installer.pl”)

zip/unzip

Windows Installer SDK, (Windows only) to create the Windows installer database

EPM, (non-Windows only) to create the non-Windows native packages

Technical Dependencies

Every native package can only be created on its native platform. This is especially important for the Windows platform, on which the Windows Installer SDK is needed to create the Windows installer database. For all other platforms this is not absolutely correct. If you compile the EPM on a Solaris computer and have a RPM for Solaris, you can also create native Linux RPMs on the Solaris computer. Vice versa this can be difficult because there is no “pkgmk” for Linux.

Competitive Analyses

RPM, PKG, and MSI are the standard installation mechanism on their respective platform. There's no UI standard for Linux and Solaris. There's no standard for GUI setup for Linux / Solaris installation.

Detailed Specification

After integration of the new scp projects (scp2 and scp2so), the new parser “pre2par.pl” and the new linker “par2script.pl” the only missing step in the packaging process from scp to native installer is the new packaging program. If this is integrated the new packaging process will be completely Perl based.

“make_installer.pl” works in two steps. In the first platform independent step all information is collected from the setup scripts, that are located in the solver. These scripts are the results of the scp projects and describe the installation set.

In the second step there is a difference for Windows and all other (non-Windows) platforms. For the Windows platform the Windows Installer SDK is used to convert the collected information into a Windows Installer database. If this database can be created the packaging process starts, in which the files from the solver are packed into Windows specific cab files. Finally a loader and the Windows Installer msi files (instmsia.exe and instmsiw.exe) are added to the installation set.

For the non-Windows platforms the collected information from the setup scripts are used to create the “list file”, which is the input for the “ESP Package Manager”. If the standard EPM is used, EPM is started with the list file and the result is a native package installation set.

Unfortunately there are two problems with the standard EPM:

the packages are created in the native package directories (Linux: /usr/src/RPMS/i586/..., Solaris: /var/spool/packages), which will lead to conflicts in a build environment.

The packages are not relocatable, which is absolutely necessary for OpenOffice.org and StarOffice

Because of these two disadvantages, EPM is used in a patched version. EPM is patched in that way, that it creates the OS specific package files, but does not start the packaging process. For Solaris the prototype, the pkginfo, ... are created. That is all. For Linux the spec file is created and all files that need to be packaged are copied locally. But then the prototype and the specfile are edited by “make_installer.pl” ot make the packages relocatable and to change the package creation directory. Additionally the installation pathes in the installation shell scripts have to be adapted to the OS specific variables.

Parameter of “make_installer.pl” (overview):

-f, optional, name of a ziplist file

-s, optional, name of a setup script file

-i, optional, name of the installation directory

-p, product name from the ziplist file, required

-l, language list, optional comma and hash separated: 01,49#01,33

-c, compiler, optional, unxsols4, wntmsci8, ...

-b, build, optional, SRC680

-m, minor, optional, m28

-pro, set, if product version

-log, saves a lot of files into the logging dir (useful for debugging)

-buildid, will be shown in the about dialog, optional

-u, unpack path, in this many helper files are created, optional

-dontunzip, really unzip archive files, otherwise only analyze zip files, optional

-msitemplate, Windows only, path to template tables, required

-msilanguage, Windows only, path to translation files for tables, required

-msifiles, Windows only, path to loader.exe, instmsia.exe and instmsiw.exe, required

-dontcallepm, non-Windows only, calling epm or stopping after creation of list file, optional

-ispatchedepm, non-Windows only, using a patched EPM, optional

-packagelist, non-Windows only, list that describes the packages that will be built, required

-copyonly, optional, this flag has to be set for all products, that are only copied by makeinstall

Detailed parameter explanation:

-f, optional, name of a ziplist file. This can be set, to overwrite the standard zip list file, which can be useful for testing reasons. If this parameter is not set, the standard file will be use, which will be the “office.lst” or “openoffice.lst” in the corresponding cvs module.

-s, optional, name of a setup script file. This can be set to overwrite the standard setup script defined in the zip list file. This is useful for testing reasons. If this parameter is not set, the standard file will be use, which is defined in the ziplist file.

-i, optional, name of the installation directory. If this is not set, the installation directory will be created from “/opt” and a subdirectory containing productname and productversion. Using this parameter can overwrite this value.

-p, product name from the ziplist file, required. This parameter is absolutely necessary to tell “make_installer.pl” which product shall be packed. Example: OfficeOSL. This is the name defined in the zip list file. “make_installer.pl” then takes all definitions of this product from the zip list file.

-l, language list, optional comma and hash separated: 01,49#01,33. This is an optional parameter to overwrite the settings in the zip list file. Using “-l 01” creates an english installation set, “-l 01, 49” an english-german installation set and “-l 01, 49#01,33” an english-german installation set and an english-french installation set. The numbers can be exchanged by the corresponding letter codes (en, de, fr) if all other changes are also done, like adapting the file names.

-c, compiler, optional, unxsols4, wntmsci8, ... This is an important parameter. “make_installer.pl” creates an installation set for the corresponding compiler. If not defined as parameter, the value is taken from the environment variable OUTPATH.

-b, build, optional, SRC680. This is an important parameter. “make_installer.pl” creates an installation set for the corresponding build. If not defined as parameter, the value is taken from the environment variable WORK_STAMP.

-m, minor, optional, m28. This is an important parameter. “make_installer.pl” creates an installation set for the corresponding minor. If not defined as parameter, the value is taken from the environment variable UPDMINOR.

-pro, set, if product version. If this parameter is set, “make_installer.pl” creates an installation set from the product version, otherwise the non product version. If not defined as parameter, the value is taken from the environment variable PROEXT.

-log, saves many files into the logging dir. This is useful for debugging process.

-buildid, will be shown in the about dialog, optional. The value is included into the bootstrap ini file. If this is not set, in the about box is written “BuildID not set”. Otherwise this value will be included.

-u, unpack path, in this many helper files are created, optional. If this parameter is set, the unpack path can be overwritten. If this parameter is not set, the unpack path will be created locally, beginning with the build, then the minor, the productname, ... This unpack path is important for the packaging process, because in this directory, the zip files are unpacked, the files with flag SCPZIP_REPLACE are created, the ini and the configuration file are created, for Windows the ddf files for packaging are created here and and and ... Just look after a packaging process into this directory.

-dontunzip, do not unzip archive files, only analyze zip files, optional. If dontunzip is not set, much disc space is needed in the unpack path. And the packaging process needs much more time. If dontunzip is set, the files are not unpacked. But therefore no working installation package can be created.

-msitemplate, Windows only, path to template tables, required. This path contains template files, that describe parts of the installation process, that cannot be described in scp projects. At the moment this are 21 files in idt format, corresponding 21 tables. Some of them are taken as they are, some still need to be localized, and to some further information is added. Most of these files describe the GUI of the installation process.

-msilanguage, Windows only, path to translation files for tables, required. Some template idt files need to be localized (ActionText, Control, Error, Property, RadioButton, UIText and the Summary Information Stream). The corresponding translation files have to be located in this aidtlanguage path. They have to have the same name as the idt files (of course they have another file extension). Both have to have the same name as the tables in the Windows Installer database.

-msifiles, Windows only, path to loader.exe, instmsia.exe and instmsiw.exe, required. There are some Windows Installer specific files, that need to be packed into the installation set. In the future this files can be copied from the solver into the installation set using ScpActions, but at the moment these files would also be added into installation sets created with the old packaging mechanism.

-dontcallepm, non-Windows only, do not call epm and stop after creation of list file, optional. If this parameter is not set, epm will be started in this packaging process. Otherwise “make_install.pl” stops after the creation of the epm list file. Then the user can use this file to create the installation sets.

-ispatchedepm, non-Windows only, using a patched EPM, optional. This parameter specifies that a patched epm is used. This patched epm is changed for Solaris and for Linux and is used for inhouse building processes first of all. The patched epm allows creation of relocatable packages and allows to define another directory, in which the packages are created.

-packagelist, non-Windows only, list that describes the packages that will be built, required. This is a very important parameter for all non-Windows platforms. The packagelist contains all information for the process of creating RPMs, Solaris-packages, ... This is important, because a product like the OpenOffice.org is described by only one setup script, the setup_osl.ins (.inf). But this product shall not be included in only one RPM or Solaris-Package. This packagelist describes the different packages that shall be created from one script. More information is available in the next paragraph “The package list file format”.

-copyonly, optional, this flag has to be set for all products, that are only copied by makeinstall. In this case only all files are copied from the solver. Only the scp items “File” and “ScpAction” are evaluated.

The package list file format (non-Windows only)

A package list has to be defined for all packaging processes of all non-Windows platforms to specify all packages that shall be created from on script file. For “make_installer.pl” the parameter “-packagelist” has to be used (see above). A package list file has the following syntax:

Each block in the package list file defines one package. A comment line begins with a “#”. Each block begins with a single line containig “Start” and ends with a singe line containing only “End”. Each line between consists of one key-value pair.

Each of the following pairs has to be defined. For the key “script” the value can be empty.

Key-Value pairs:

module = “gid_Module_Optinal_Cde” : This module and all submodules are packed into one package. All module above this module do not contain this module (for example gid_Module_Root). Therefore such a definition can also be used simply to exclude a specified module.

script = “shellscripts.txt” : This file contains the shellscripts that are included into the epm file list and then into the platform specific installation files, for example the spec file for Linux. The value can be empty.

solarispackagename = “SUNWoo20”: The package name for Solaris packages

packagename = “openoffice20kde”: The package name for all non-Solaris packages

copyright = “This is the copyright text” : Included into the epm list file header. This text is shown if the package is queried (Example: “rpm -q”)

vendor = “Here is the vendor” : Included into the epm list file header. This text is shown if the package is queried (Example: “rpm -q”)

description = “A little description” : Included into the epm list file header. This text is shown if the package is queried (Example: “rpm -q”)

The complete product is packed into only one package, if only one block is defined, in which the module line contains:

module=”gid_Module_Root”

What makes “make_installer.pl”?

“make_install.pl” is divided in two parts, which are again parted in two parts. Therefore there are four parts that shall be described now shortly:

Part 1: The platform independent part

Part 1a: The language independent part:

Checking the parameter

Checking used programs and environment variables

Reading the zip list file (include pathes, variable names, script names, ...)

Reading the setup script

Replacing all variables in setup script

Collecting and analyzing all items from setup script (Directories, Files, Profiles, ...)

Part 1b: The language dependent part:

Collecting all languages for one product

Resolving all include pathes

Resolving all language dependent items, like Files, Profiles, ...

Removing items without correct language definition (Files only for special languages)

Searching the source path for all defined Files

Adding license files to destination directory (formerly done by a basic script)

Unpack all files with flag ARCHIVE

Replace variables in files with flag SCPZIP_REPLACE

Replace variables in files with flag PATCH_SO_NAME

Create services.rdb file from all UNO_COMPONENT libraries

Resolving the directory tree, analyze directories with CREATE flag

Searching the source path for ScpActions

Create relative links for all shortcuts with flag RELATIVE

Create all ini files that are defined by Profiles and ProfileItems in scp projects

Create all configuration files that are defined by ConfigurationItems in scp projects

Part 2: The platform dependent part

Part 2a: All non-Windows platforms

Analyzing all packages defined in the package list

The next steps are repeated for each package defined in the packagelist

Strip the files that are not stripped yet

Create the epm file list

Call EPM

The final steps only for patched epm and only for Linux and Solaris:

Make packages relocatable

Modify the shellscrips for relocatable packages

Change the directory in which the packages are created

Start the packaging process (pkgmk, rpm)

Part 2b: The Windows platform

Create idt files (tables in the Windows installer database) with setup script information

Create File table

Create Directory table

Create Registry table

Create Component table

Create Feature table

Create FeatureComponent table

Create Media table

Create Font table

Create Shortcut table

Create Icon table

Create RemoveFile table

Create CreateFolder table

Modify the Upgrade table

Copy the template tables and localize them

Include language selection dialog for multilingual installation sets

Add the license file into the Control table

Update the Property table

Include the CustomAction for the configuration in the tables

Create the msi database (uses Windows Installer SDK)

Write the Summary Information Stream into the database (uses WI SDK)

Create Transforms for multilingual installation sets (uses WI SDK)

Copy loader.exe, instmsia.exe and instmsiw.exe into installation sets

Create ddf files for packaging

Pack the installation set consisting of cab-files (uses WI SDK)

Technical aspects of the Perl packaging process

The new packaging progam is located in the main program “make_install.pl” and its Perl modules located in the subdirecotories “installer” and “installer/windows”. Here is a short description of the different files (modules in alphabetical order):

make_install.pl : The main program, leads to all steps shown in the proceeding paragraph

installer/archivefiles.pm : Functions for analyzing all files with flag ARCHIVE

installer/configuration.pm : Functions to convert the ConfigurationItems into configuration files

installer/control.pm : Controller functions, for environment variables, required products and syntax of the packagelist

installer/converter.pm : Often uses converter functions (Example: array to hash)

installer/copyproject.pm : Functions that are used for copy-only projects (parameter: -copyonly)

installer/environment.pm : Functions in which the environment variables are checked and global variables are set

installer/epmfile.pm : Collection of functions, which are used to create and edit the epm file list

installer/existence.pm : Often used functions that check the existence of defined arguments

installer/exiter.pm : The program exit function, that logs data before exiting

installer/files.pm : System functions for reading and writing files

installer/globals.pm : Global definitions for the complete program

installer/languages.pm : Functions that handle language specific tasks

installer/logger.pm : Functions to simplify the logging mechanism

installer/packagelist.pm : Functions that analyze the language list, but also functions that collect all information for each package. The scp information is transferred into the epm list file information.

installer/parameter.pm : All functions to get all parameters, to control them, to set global variables and to write the parameter to the console.

installer/pathanalyzer.pm : Some functions to analyze a path

installer/pathfile.pm : obsolete

installer/profiles.pm : Functions to convert scp Profiles and ProfileItems into files.

installer/remover.pm : Often used functions to remove parts of strings (Example: ending spaces)

installer/scppatchsoname.pm : Functions to patch all files with flag PATCH_SO_NAME

installer/scpzipfiles.pm : Functions to patch all files with flag SCPZIP_REPLACE

installer/scriptitems.pm : Collection of functions to collects, change or delete the ScriptItems like Files, ShortCuts, ...

installer/servicesfile.pm : Functions for the creation of the services.rdb file

installer/setupscript.pm : Functions to work with the setup script

installer/sorter.pm : Collection of functions for sorting

installer/strip.pm : Collection of functions for stripping files

installer/systemactions.pm : Functions like the creation of directories or the copying of files

installer/ziplist.pm : Functions that work with the zip list file

installer/windows/component.pm : Functions to create the Windows Installer Component table

installer/windows/createfolder.pm : Functions to create the Windows Installer CreateFolder table

installer/windows/directory.pm : Functions to create the Windows Installer Directory table

installer/windows/feature.pm : Functions to create the Windows Installer Feature table

installer/windows/featurecomponent.pm : Functions to create the Windows Installer FeatureComponent table

installer/windows/file.pm : Functions to create the Windows Installer File table

installer/windows/font.pm : Functions to create the Windows Installer Font table

installer/windows/icon.pm : Functions to createa the Windows Installer Icon table

installer/windows/idtglobal.pm : Collection of functions that are used for the creation of the idt table files.

installer/windows/language.pm : Functions that handle with Windows specific language settings

installer/windows/media.pm : Functions to create the Windows Installer Media table

installer/windows/msiglobal.pm : Collection of function that are used to create and modify the Windows Installer database

installer/windows/property.pm : Functions to update the Windows Installer Property table

installer/windows/registry.pm : Functions to create the Windows Installer Registry table

installer/windows/removefile.pm : Functions to create the Windows Installer RemoveFile table

installer/windows/shortcut.pm : Functions to create the Windows Installer Shortcut table

installer/windows/upgrade.pm : Functions to create the Windows Installer Upgrade table

Error Conditions

ESP Package Manager (EPM)

The ESP Package Manager (EPM) is a tooling provided under the GPL by Easy Software Products (ESP) that creates installation packages for many different Unix platforms and MacOS X from a unified file format (list file). This list file contains the complete information of files to be packaged and scripts to be executed and is a mixture of a Linux spec file and a Solaris prototype and a pkginfo file.

Additional functionality of EPM:

Support for many OOo platforms that we do not support like Debian, MacOS X, ...

An additional EPM specific installation format (no native installer), that offers a patch mechanism and a graphical UI.

Future Tasks

Add the CNP SDK as GUI installer for StarOffice RPMs and Solaris-Packages (another CWS).

Legal Issues

Windows Installer, the legal review is still pending.

Hosting parts of Windows Installer database tables on OpenOffice.org

EPM (Open Source)

Windows Installer SDK

Notes

References

Documentation of scp projects:

http://staroffice-doc.germany.sun.com:8080/Projects/Common/Documentation/Developer_Docs/How_to_Scp/how_to_scp.html

ESP Package Manager:

http://www.easysw.com/epm/

Windows Installer SDK:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/msi/setup/about_windows_installer.asp

Windows Installer:

Inside Windows Installer, Andreas Kerl, Microsoft Press, 2003

The Official InstallShield for Windows Installer Developer's Guide, Bob Baker, M&T Books, 2000

<Native Installer Packaging Process>

Page 12

2003-10-07T15:20:14