Current version of rpm.rte is 4.15.1.0 Beginning sections (INTRODUCTION, ISSUES WITH OLDER APPROACH, CHANGES IN NEWER rpm.rte, LIMITATIONS WITH NEWER SOLUTIONS, IMPORTANT NOTE, KNOWN ISSUES) talks about the major changes introduced from rpm.rte-4.13.0.4. Please check "Change History" section to know more detailes of the changes done so far. INTRODUCTION ============ rpm.rte older than 4.13.0.4 ships binaries and libraries in /usr/opt/freeware/lib and /usr/opt/freeware/bin paths. rpm.rte-4.15.1.0 is dependent on libraries from the below mentioned minimal versioned rpm packages. rpm.rte-3.0.5.x has a slightly lesser dependency than rpm.rte-4.15.1.0. So as to mention, rpm.rte ships libraries extracted from below mentioned rpm packages. file-5.36 lua-5.3.4 popt-1.16 readline-8.0 gettext-0.19.8.1 zlib-1.2.11 nspr-4.21 nss-3.45.0 sqlite-3.32.1 db-4.8.24 Also, other binary and libraries are shipped from these below rpm packages. These tools are required in case of rpm packages build etc.. bzip2-1.0.8 gzip-1.10 texinfo-6.6 (info) patch-2.7.6 ISSUES WITH OLDER APPROACH ========================== The problem with the older approach was that. 1. If any rpm package is installed, which is providing same library as what rpm.rte is installed, then the library from rpm package overwrites the library installed by rpm.rte fileset. Side effects of this is that, the rpm command might fail to load if the overwritten library from rpm package isn't compatible with what rpm.rte needs. This can cause other AIX commands like oslevel and lslpp to fail. 2. If any rpm package is installed, which is providing same library as what rpm.rte is installed then users can't remove this package. AIX-rpm, a virtual rpm package adds a dependency on these packages as the libraries are required to run rpm commands. AIX-rpm is a virtual package which reflects what has been installed on the system through rpm.rte fileset. 3. If with rpm.rte-3.0.5.x, users install non toolbox repository rpm packages or in-house built rpm packages like gettext, db etc., then at his point all users applications are in working condition. Later, if update to latest rpm.rte-4.13.0.x happens either through TL update or manual update to rpm.rte-4.13.0.x or to rpm.rte-3.0.5.x, then rpm.rte overwrites the libraries installed by a users rpm package if they are same. So, later users application can fail if library what rpm.rte provides is incompatible with what the users application requires. CHANGES IN NEWER rpm.rte (4.13.0.4 onwards) =========================================== Newer solution has been implemented to resolve the issues mentioned above. Here are the highlights of the new changes. -> Newer rpm.rte ships it's required binaries and libraries under the path /usr/opt/rpm, which is totally isolated path than the earlier /opt/freeware paths. -> AIX-rpm doesn't have dependency on packages like gettext, db, readline etc., users can install and uninstall these packages and rpm command works fine. -> With rpm-3, if users installed packages like gettext, db etc.. then migrating to a newer rpm.rte won't overwrite what has been installed by these rpm packages. Users installed libraries and binaries will be taken backup and restored so that their environment won't be effected even if users installs packages from perzl or bull. Users application won't be effected and rpm command works without any issues. -> If no correspnding rpm package is installed which provides the same library/binary as rpm.rte, then the symbolic links will be created in /usr/opt/freeware which points to newer installed library/binary which are in /us/opt/rpm. -> Package building won't be effect, as the build, source paths, prefix, libpath, binpath are controlled using a macros file. So, when we build rpm package using 4.13.0.4, we should be getting the rpm packages with binaries and libraries path as our default __prefix path /opt/freeware. LIMITATIONS WITH NEWER SOLUTIONS ================================ With newer rpm.rte users can remove packages like gettext, db etc., as AIX-rpm doesn't have any dependency on them. However, after removing, binaries and libraries from these packages would be removed, which might be required for some other packages. This is because of the assumption that some binaries and libraries will be present in the system by default from the rpm.rte. One such example is. In some of rpm packages /sbin/install-info command is used in post install scripts. /sbin/install-info is provided by rpm.rte and also an info rpm package. If we remove info rpm package after migration to 4.13.0.4 then /sbin/install-info would be removed and some packages installation might fail later due to missing /sbin/install-info. One solution we are trying is to fix the info rpm package and not to overwrite what rpm.rte provides. Smilarly for other packages. As we are creating a symbolic links in /usr/opt/freeware path to newer path /usr/opt/rpm, if no rpm package is installed. Later, if rpm package is installed which overwrites the symbolic created in /usr/opt/freeware, then again if that rpm package is removed, we won't be having a symbolic links. If we again want symbolic links to be present without installing an rpm package, then run the script "/usr/opt/rpm/bin/recreate_opt_bin_lib_links" which will try to recreate the links. IMPORTANT NOTE ============== As mentioned in limitation section, please make sure to check if the rpm packages being removed is required by any other rpm packages. KNOWN ISSUES ============ As mentioned in limitation sections we might see error while installing rpm packages if we removed the installed packages like info. As per the new approach, though removing packages won't effect any rpm functionality they might causes issues with other rpm packages. 1. Removing installed info rpm packages might through error about missing /sbin/install-info while installing rpm packages which uses install-info command in pre/post install script. 2. Removing installed gzip might through error about missing gzip command if gzip command is used in pre/post install script. Similarly removing packages like gettext, db, etc. can cause issues. Change History ============== Here is a brief summary of changes done for the rpm 4.13.0.x over time. 4.13.0.1 -------- - Update to latest version 4.13.0.1. 4.13.0.2 -------- - Fix issues caused during migration from rpm.rte-3.0.5.x to rpm.rte-4.13.0.1. See technote http://www-01.ibm.com/support/docview.wss?uid=isg3T1027160 4.13.0.3 -------- - Use intermediate rpm-4.0 to convert db1 to db3 database format. With this change, no pre install scripts are used to query the package installed with rpm-3, create a virtual package, and then install them to rpm-4 database. Lot of issues been reported when we try to migrate packages manually using pre install scripts to a new database. With the use of rpm-4.0 to convert db1 to db3 format, running single command converts to a new database format. 4.13.0.4 -------- - Change install path from /usr/opt/freeware to /usr/opt/rpm for the reasons mentioned in beginning sections. 4.13.0.5 -------- - Unset some environment variables in /usr/bin/rpm to prevent rpm core: Here are the list of variables. AIXTHREAD_MNRATIO AIXTHREAD_SCOPE XPG_UNIX98 - Fix rpmbuild issue with 64 bit only rpm packages: During rpm package build, find-provides is used to check what are all the library a particular rpm package provides. But, if we are building only 64-bit version of an rpm package then find-provides doesn't shows any library being provided by an rpm package which causing issue in dependency resolution. Changes done in /usr/opt/rpm/lib/rpm/find-provides to understand both 32 and 64 bit builds. 4.13.0.6 -------- - Fix rpm_share locked files causes oslevel hangs: If their is a corruption in the rpm database, and after the corruption first rpm process acquires the shared lock and then calls select call with timeout value of 6 secs. So, before the first process timeout expires and release the lock, if another rpm process is started then new rpm process will be waiting the for the lock. Sequence always here is, acquire the lock (after first process timeout expires and released the lock) and then wait for sometime using timeout value as we have rpm database corrupted. If we add more and more process before timeout expires, processes completion gets delayed and more and more processes keep on running eventaully causing the hang. - Fix RPM coredumps for non-root users: rpm core dumps when LDR_CNTRL=MAXDATA=0xb0000000@DSA is set for non-root user. When we set maxdata to 0xb and call mmap then os is returning the mmap address to match the page boundary and the address we always get is ffffe000. When we run rpm query command, db is trying to open database files __db*. If user doesn't have permission to open these files then it goes and tries to read the files like Packages, Name etc.. The file size we had for the Name is exactly 8192 which is off two pages. So in db, check was being done to see if we aren't mixing the mmap address with other buffer address. The check was mmap address+filelength which is ffffe000+8192, and it is 0, and the check was failing hence we were wrongly assuming mmap address as a buffer address and the coredump. 4.13.0.7 -------- - Fails to load rpm command during migration to latest AIX version. If we migrate AIX version which has rpm-3.0.5.x and then migrate to AIX where we have rpm.rte-4.13.0.4 and higher version, then in some cases we can see rpm command failing to load. Newer rpm looks for libraries in /usr/opt/rpm/lib path. This version of rpm.rte doesn't overwrite some of libraries already installed under /opt/freeware/lib to prevent breaking compatibility of already working packages. So, if we have some older incompatible libraries in /opt/freeware/lib and set the LIBPATH variable then rpm might fails to load. 4.13.0.8 -------- - Create a required binary and library symlink in /usr/opt/freeware pointing /usr/opt/rpm Newer rpm.rte is shipping it's required files and library in /usr/opt/rpm path. In older versions of rpm.rte, files were being shipped in /usr/opt/freeware path. To maintain the backward compatibility when we install newer rpm.rte we still need to find a way to provide the library and binaries which were used to ship in /usr/opt/freeware path. Symlink should be created only if no rpm package is installed or file isn't present in the systmem and not owned by an rpm package. With creating a symlink. 1) we are making sure that we aren't breaking the existing environment by overwriting or create symlink for the file which is already present in the system and owned by an rpm package. 2) Files will be owned by only an rpm package. Now, single file won't owned by both rpm.rte lpp and an rpm package. 3) No lppchk error for rpm.rte if rpm package is removed. 4) No need to ship the same file in multiple places, one copy is enough and create a symlink which will indicate it's from the rpm.rte fileset. 4.13.0.9 -------- - Add CVE fixes CVE-2017-7500 CVE-2017-7501. - Provide script /usr/opt/rpm/bin/recreate_opt_bin_lib_links to recreate a symbolic link if they removed by an rpm package. - Change hardcoded build related paths in macros file. Some of the build related paths are hardcoded instead of referencing them through the %{_topdir}. - Update /usr/sbin/updtvpkg script to directly use /usr/opt/rpm/lib/rpm/find-provides - Make default fuzz facotr to 2 instead of 0 for the patch command This change is in /usr/opt/rpm/lib/rpm/macros file. - Update file version to 5.36 While building packages like golang rpmbuild hangs due to bug in current version of file. With file-5.36 no hang issue is seen with rpmbuild. 4.13.0.10 --------- - Fix nspr library thread priority change issue. https://bugzilla.mozilla.org/show_bug.cgi?id=871064 rpm would hang or take very long time to complete the execution if the CPU utilization is more than 60-70% by high priority threads. This is caused as nspr library was calling pthread_setschedparam with priority 1 and default scheduling policy (0). This was making rpm to run with very less priority and causing delay in execution. 4.13.0.11 --------- - Rebuild rpm.rte with a missing signature for TSD with secureboot mode support. =================================================================================== 4.15.1.0 -------- - Update to newer version 4.15.1.0 4.15.1.1 -------- - Update sqlite to 3.33.0 to include CVE fixes. 4.15.1.2 -------- - Add fix for rpm --verify issue. rpm -V gives mode error for some of the directories. For example. .M....... /opt/freeware/64 .M....... /opt/freeware/64/lib rpm internally uses the file mode flag as rpm_mode_t which assumes the size is uint16_t, but AIX mode_t is unsigned long. rpm uses the mode_t value return from stat calls and uses only 16-bits. So, if mode_t is used directly without typecast to rpm_mode_t then mode value differs from what is rpm expects. - rpm format issue for octal and hex filemode query When we query a rpm package with filemode of either octal or hex format then we always get a 0 filemode instead of the actual filemode value. # rpm --qf '[%{FILEMODES:octal} %{FILENAMES}\n]' -q bash 0 /bin/bash 0 /bin/bash_32 This causes to set all permission of a file to 0 when we use rpm --setperms. Since rpm by default is 32-bit and the value returned by one of function is 64-bit long. So while printing the actual value in varargs functions the actual value passed is 64-bit instead of 32-bit. 4.15.1.3 -------- - Change soft links in /usr/bin and /usr/lib which are pointing to /usr/opt/freeware/bin and /usr/opt/freeware/lib to /usr/opt/rpm/bin and /usr/opt/rpm/lib if no rpm package is already owning it. - Update file version to 5.39 which has important fix related to strndup. This avoids malloc failure during some packages build. memory alloc (1649664 bytes) returned NULL - Ship rebuilt zlib which has better optimization.