Current version of rpm.rte is 4.15.1.0
Beginning sections (INTRODUCTION, ISSUES WITH OLDER APPROACH,
CHANGES IN NEWER rpm.rte, LIMITATIONS WITH NEWER SOLUTIONS, IMPORTANT NOTE,
KNOWN ISSUES) talks about the major changes introduced from rpm.rte-4.13.0.4.
Please check "Change History" section to know more detailes of the changes done
so far.
INTRODUCTION
============
rpm.rte older than 4.13.0.4 ships binaries and libraries in
/usr/opt/freeware/lib and /usr/opt/freeware/bin paths.
rpm.rte-4.15.1.0 is dependent on libraries from the below mentioned minimal
versioned rpm packages. rpm.rte-3.0.5.x has a slightly lesser dependency than
rpm.rte-4.15.1.0. So as to mention, rpm.rte ships libraries extracted from
below mentioned rpm packages.
file-5.36
lua-5.3.4
popt-1.16
readline-8.0
gettext-0.19.8.1
zlib-1.2.11
nspr-4.21
nss-3.45.0
sqlite-3.32.1
db-4.8.24
Also, other binary and libraries are shipped from these below rpm packages.
These tools are required in case of rpm packages build etc..
bzip2-1.0.8
gzip-1.10
texinfo-6.6 (info)
patch-2.7.6
ISSUES WITH OLDER APPROACH
==========================
The problem with the older approach was that.
1. If any rpm package is installed, which is providing same library as what
rpm.rte is installed, then the library from rpm package overwrites the
library installed by rpm.rte fileset.
Side effects of this is that, the rpm command might fail to load if the
overwritten library from rpm package isn't compatible with what rpm.rte
needs. This can cause other AIX commands like oslevel and lslpp to fail.
2. If any rpm package is installed, which is providing same library as what
rpm.rte is installed then users can't remove this package.
AIX-rpm, a virtual rpm package adds a dependency on these packages as the
libraries are required to run rpm commands. AIX-rpm is a virtual package
which reflects what has been installed on the system through rpm.rte fileset.
3. If with rpm.rte-3.0.5.x, users install non toolbox repository rpm packages or
in-house built rpm packages like gettext, db etc., then at his point all
users applications are in working condition. Later, if update to latest
rpm.rte-4.13.0.x happens either through TL update or manual update to
rpm.rte-4.13.0.x or to rpm.rte-3.0.5.x, then rpm.rte overwrites the libraries
installed by a users rpm package if they are same.
So, later users application can fail if library what rpm.rte provides is
incompatible with what the users application requires.
CHANGES IN NEWER rpm.rte (4.13.0.4 onwards)
===========================================
Newer solution has been implemented to resolve the issues mentioned above.
Here are the highlights of the new changes.
-> Newer rpm.rte ships it's required binaries and libraries under the path
/usr/opt/rpm, which is totally isolated path than the earlier /opt/freeware
paths.
-> AIX-rpm doesn't have dependency on packages like gettext, db, readline etc.,
users can install and uninstall these packages and rpm command works fine.
-> With rpm-3, if users installed packages like gettext, db etc.. then migrating
to a newer rpm.rte won't overwrite what has been installed by these rpm
packages. Users installed libraries and binaries will be taken backup and
restored so that their environment won't be effected even if users installs
packages from perzl or bull. Users application won't be effected and rpm
command works without any issues.
-> If no correspnding rpm package is installed which provides the same
library/binary as rpm.rte, then the symbolic links will be created in
/usr/opt/freeware which points to newer installed library/binary which are
in /us/opt/rpm.
-> Package building won't be effect, as the build, source paths, prefix,
libpath, binpath are controlled using a macros file. So, when we build rpm
package using 4.13.0.4, we should be getting the rpm packages with
binaries and libraries path as our default __prefix path /opt/freeware.
LIMITATIONS WITH NEWER SOLUTIONS
================================
With newer rpm.rte users can remove packages like gettext, db etc., as AIX-rpm
doesn't have any dependency on them. However, after removing, binaries and
libraries from these packages would be removed, which might be required for
some other packages. This is because of the assumption that some binaries and
libraries will be present in the system by default from the rpm.rte.
One such example is.
In some of rpm packages /sbin/install-info command is used in post install
scripts. /sbin/install-info is provided by rpm.rte and also an info rpm package.
If we remove info rpm package after migration to 4.13.0.4 then /sbin/install-info
would be removed and some packages installation might fail later due to
missing /sbin/install-info.
One solution we are trying is to fix the info rpm package and not to overwrite
what rpm.rte provides. Smilarly for other packages.
As we are creating a symbolic links in /usr/opt/freeware path to newer path
/usr/opt/rpm, if no rpm package is installed. Later, if rpm package is installed
which overwrites the symbolic created in /usr/opt/freeware, then again if that
rpm package is removed, we won't be having a symbolic links.
If we again want symbolic links to be present without installing an rpm
package, then run the script "/usr/opt/rpm/bin/recreate_opt_bin_lib_links"
which will try to recreate the links.
IMPORTANT NOTE
==============
As mentioned in limitation section, please make sure to check if the rpm
packages being removed is required by any other rpm packages.
KNOWN ISSUES
============
As mentioned in limitation sections we might see error while installing rpm
packages if we removed the installed packages like info.
As per the new approach, though removing packages won't effect any rpm
functionality they might causes issues with other rpm packages.
1. Removing installed info rpm packages might through error about missing
/sbin/install-info while installing rpm packages which uses install-info
command in pre/post install script.
2. Removing installed gzip might through error about missing gzip command
if gzip command is used in pre/post install script.
Similarly removing packages like gettext, db, etc. can cause issues.
Change History
==============
Here is a brief summary of changes done for the rpm 4.13.0.x over time.
4.13.0.1
--------
- Update to latest version 4.13.0.1.
4.13.0.2
--------
- Fix issues caused during migration from rpm.rte-3.0.5.x to
rpm.rte-4.13.0.1.
See technote http://www-01.ibm.com/support/docview.wss?uid=isg3T1027160
4.13.0.3
--------
- Use intermediate rpm-4.0 to convert db1 to db3 database format.
With this change, no pre install scripts are used to query the package
installed with rpm-3, create a virtual package, and then install them
to rpm-4 database.
Lot of issues been reported when we try to migrate packages manually
using pre install scripts to a new database.
With the use of rpm-4.0 to convert db1 to db3 format, running single
command converts to a new database format.
4.13.0.4
--------
- Change install path from /usr/opt/freeware to /usr/opt/rpm for the
reasons mentioned in beginning sections.
4.13.0.5
--------
- Unset some environment variables in /usr/bin/rpm to prevent rpm core:
Here are the list of variables.
AIXTHREAD_MNRATIO
AIXTHREAD_SCOPE
XPG_UNIX98
- Fix rpmbuild issue with 64 bit only rpm packages:
During rpm package build, find-provides is used to check what are all the
library a particular rpm package provides. But, if we are building only
64-bit version of an rpm package then find-provides doesn't shows any
library being provided by an rpm package which causing issue in dependency
resolution.
Changes done in /usr/opt/rpm/lib/rpm/find-provides to understand both 32
and 64 bit builds.
4.13.0.6
--------
- Fix rpm_share locked files causes oslevel hangs:
If their is a corruption in the rpm database, and after the corruption
first rpm process acquires the shared lock and then calls select call
with timeout value of 6 secs.
So, before the first process timeout expires and release the lock,
if another rpm process is started then new rpm process will be waiting
the for the lock. Sequence always here is, acquire the lock (after first
process timeout expires and released the lock) and then wait for sometime
using timeout value as we have rpm database corrupted.
If we add more and more process before timeout expires, processes
completion gets delayed and more and more processes keep on running
eventaully causing the hang.
- Fix RPM coredumps for non-root users:
rpm core dumps when LDR_CNTRL=MAXDATA=0xb0000000@DSA is set for non-root
user.
When we set maxdata to 0xb and call mmap then os is returning the mmap
address to match the page boundary and the address we always get is
ffffe000. When we run rpm query command, db is trying to open database
files __db*. If user doesn't have permission to open these files then it
goes and tries to read the files like Packages, Name etc..
The file size we had for the Name is exactly 8192 which is off two pages.
So in db, check was being done to see if we aren't mixing the mmap address
with other buffer address. The check was mmap address+filelength which is
ffffe000+8192, and it is 0, and the check was failing hence we were wrongly
assuming mmap address as a buffer address and the coredump.
4.13.0.7
--------
- Fails to load rpm command during migration to latest AIX version.
If we migrate AIX version which has rpm-3.0.5.x and then migrate to AIX
where we have rpm.rte-4.13.0.4 and higher version, then in some cases
we can see rpm command failing to load.
Newer rpm looks for libraries in /usr/opt/rpm/lib path.
This version of rpm.rte doesn't overwrite some of libraries already
installed under /opt/freeware/lib to prevent breaking compatibility of
already working packages.
So, if we have some older incompatible libraries in /opt/freeware/lib and
set the LIBPATH variable then rpm might fails to load.
4.13.0.8
--------
- Create a required binary and library symlink in
/usr/opt/freeware pointing /usr/opt/rpm
Newer rpm.rte is shipping it's required files and library in /usr/opt/rpm
path. In older versions of rpm.rte, files were being shipped in
/usr/opt/freeware path. To maintain the backward compatibility when we
install newer rpm.rte we still need to find a way to provide the library
and binaries which were used to ship in /usr/opt/freeware path.
Symlink should be created only if no rpm package is installed or file isn't
present in the systmem and not owned by an rpm package.
With creating a symlink.
1) we are making sure that we aren't breaking the existing environment by
overwriting or create symlink for the file which is already present in
the system and owned by an rpm package.
2) Files will be owned by only an rpm package. Now, single file won't owned
by both rpm.rte lpp and an rpm package.
3) No lppchk error for rpm.rte if rpm package is removed.
4) No need to ship the same file in multiple places, one copy is enough and
create a symlink which will indicate it's from the rpm.rte fileset.
4.13.0.9
--------
- Add CVE fixes CVE-2017-7500 CVE-2017-7501.
- Provide script /usr/opt/rpm/bin/recreate_opt_bin_lib_links to recreate a
symbolic link if they removed by an rpm package.
- Change hardcoded build related paths in macros file.
Some of the build related paths are hardcoded instead of referencing them
through the %{_topdir}.
- Update /usr/sbin/updtvpkg script to directly use
/usr/opt/rpm/lib/rpm/find-provides
- Make default fuzz facotr to 2 instead of 0 for the patch command
This change is in /usr/opt/rpm/lib/rpm/macros file.
- Update file version to 5.36
While building packages like golang rpmbuild hangs due to bug in
current version of file. With file-5.36 no hang issue is seen with
rpmbuild.
4.13.0.10
---------
- Fix nspr library thread priority change issue.
https://bugzilla.mozilla.org/show_bug.cgi?id=871064
rpm would hang or take very long time to complete the execution if the CPU
utilization is more than 60-70% by high priority threads.
This is caused as nspr library was calling pthread_setschedparam with
priority 1 and default scheduling policy (0).
This was making rpm to run with very less priority and causing delay in
execution.
4.13.0.11
---------
- Rebuild rpm.rte with a missing signature for TSD with secureboot mode support.
===================================================================================
4.15.1.0
--------
- Update to newer version 4.15.1.0
4.15.1.1
--------
- Update sqlite to 3.33.0 to include CVE fixes.
4.15.1.2
--------
- Add fix for rpm --verify issue.
rpm -V gives mode error for some of the directories.
For example.
.M....... /opt/freeware/64
.M....... /opt/freeware/64/lib
rpm internally uses the file mode flag as rpm_mode_t which assumes
the size is uint16_t, but AIX mode_t is unsigned long.
rpm uses the mode_t value return from stat calls and uses only 16-bits.
So, if mode_t is used directly without typecast to rpm_mode_t then mode
value differs from what is rpm expects.
- rpm format issue for octal and hex filemode query
When we query a rpm package with filemode of either octal or hex format
then we always get a 0 filemode instead of the actual filemode value.
# rpm --qf '[%{FILEMODES:octal} %{FILENAMES}\n]' -q bash
0 /bin/bash
0 /bin/bash_32
This causes to set all permission of a file to 0 when we use rpm --setperms.
Since rpm by default is 32-bit and the value returned by one of function
is 64-bit long. So while printing the actual value in varargs functions the
actual value passed is 64-bit instead of 32-bit.
4.15.1.3
--------
- Change soft links in /usr/bin and /usr/lib which are pointing to
/usr/opt/freeware/bin and /usr/opt/freeware/lib to /usr/opt/rpm/bin and
/usr/opt/rpm/lib if no rpm package is already owning it.
- Update file version to 5.39 which has important fix related to strndup.
This avoids malloc failure during some packages build.
memory alloc (1649664 bytes) returned NULL
- Ship rebuilt zlib which has better optimization.