Make API of libisofs and libburn compatibly expandable

[scdbackup]

Both libs must be made ready for having a stable SONAME. Designated are libisofs.so.6 and libisoburn.so.1.

Dynamically linked applications cannot rely only on SONAME, which is enforced by the linker at program start. They also have to check for outdated revision by

*_version(&major, &minor, &micro);

I.e. they need to know the library versions which where current at build time.

The stabilization work is underway for both libs and also xorriso is reviewed for convenient but incompatible changes before going public.

---

This ticket is to collect progress reports and the final oks.

---

BTW: i thought we go for libisofs-0.6.1 and not for 0.6.0.

.1 would be a development version with growing API.

.0 would be a release version with frozen API (only bug fixes allowed)

It might last a while until libisofs can issue a pare micro revision.

[pygi]

If we would go with some more sane versioning scheme then we should use odd version numbers for development versions (0.5.x, 0.7.x) and even numbers for stable versions.

[vreixo]

Commited revision 291, with configure.ac ready for 0.6.0 and library libisofs.6.0.0. But now I agree of the idea of odd version numbers for development versions. What about 0.5.x?

[pygi]

What do you say Thomas? Would you agree with switching to odd/even versioning scheme?

[scdbackup]

Odd and pare in my scheme are applied to the "micro" numbers, not to the "minor".

libburn-0.4.2 is a stable release with the promise to provide bugfixes but not to change the API.

libburn-0.4.3 is a development revision with the promise to provide bugs and fixes, no restriction to enlarge the API, permission to revoke API changes which were introduced in the same 0.4.3 revision.

What big unstable periods do you want to have in order to fill a whole 0.5.x range ? And then in 0.6.x ? What happens there over the course of .x ?

[pygi]

libburn-0.4.3 however will never be released. If we used the way suggested above, it would allow us to have two releases in the wild published in parallel (one from stable series, one from unstable series). Unstable series can break api and do whatever any time, while stable is only bugfixes and such. In stable series example (i.e. 0.6.x) we get a chance to fix bugs and problems we encounter, either by writing new patches or backporting from unstable. After some time, our unstable becomes stable and we go all over again.

[scdbackup]

This will break the runtime .so compatibility check i introduced only a few days ago.

Should we not give the concept a chance that worked well for 18 months of libburn development ? This concept allowed me to establish the runtime check far back into the times when i had no clue that i would need one at all.

So a change in revision numbering must either provide as much brainwork as the freshly established one, or it has to be regarded as regression.

[vreixo]

The .x in 0.5.x is, in my opinion, a way to release manteinance bug fixes of a released version. For example, if after releasing 0.5.0 we found a bug, it could be fixed and backported to existent releases, thus we release 0.5.1. That way we can provide manteinance of old releases, of course during a short time (maybe 6 months?).

[pygi]

Good argumentation there Thomas. Current way has served us well for a long time. Please gimme little time to think about it, and I'll see if I can come up with solution, and some solid reasons why we should stay the current way or go for nwe.

[pygi]

Vreixo, we can do the same even with 0.6.x. Maintaining old releases isn't that good, especially when you have such fast-developing software as libburnia subprojects.

[scdbackup]

Maintaining old releases is no fun.

I do it rather like this:

New wishes are fulfilled in 0.X.odd.

Bugs are fixed in 0.X.pare until the next .pare gets released. Bug fixes have the same version but the tarball gets a .pl## suffix.

[scdbackup]

Constraints for a versioning system:

It must be described in configure.ac .

It must include a rule for LT_CURRENT, LT_AGE, LT_REVISION. SONAME is LT_CURRENT - LT_AGE and has to be stable for several months or better years.

It must provide major.minor.micro number suitable for the tarball name and for an age comparison at runtime.

The age comparison is for now defined by

dnl Normally one can allow a program to run with a library which passed the
dnl linker SONAME test and which is not older than the library it was
dnl developed for. Library2 is younger than library1 if:
dnl   MAJOR2>MAJOR1 || (MAJOR2==MAJOR1 && 
dnl                     (MINOR2>MINOR1 || (MINOR2==MINOR1 && MICRO2 > MICRO1)))

But if the library provides an own comparison function

int lib_is_young_enough(int app_provided_major, int app_provided_minor, int app_provided_micro);

then the library can use any own scheme that can be implemented on three small integers.

[vreixo]

Ok,

0.6.1 will be next libisofs version.

[scdbackup]

We are not done here yet.

First we have to deliver the compatibly expandable APIs. At least libisoburn is not safe yet.

---

So 0.6.1 will be a development snapshot, not to be used with dynamic linking unless one is sure to have app and library matching exactly like at build time.

As soon as libisofs is a bit slower in API enhancements, there should be a 0.X.pare , which is is recommended as foundation of app programming.

Bugs in this 0.X.pare will either be very small or lead to a timely release of a new higher numbered 0.X.pare which gets derived from 0.X.odd, nevertheless.

So we do not have two independent development strains with constant backporting effort.

Two strains would make .so life very hard, i figured out meanwhile. They would need two different SONAME, at least.

[scdbackup]

I evaluated the potential API compatibility problems in libisoburn:

---

There will be a function

  isoburn_is_compatible(major, minor, micro, flag)

which allows an app to make a simple compatibility test by submitting its idea of libisoburn revision. So libisoburn will be free to switch to a different version numbering scheme, if this ever seems appropriate.

I propose the same function for libisofs.

In libburn its value is diminished by the fact that SONAME 4 had versions which do not offer this call. So bun_version() needs to be inquired throughout libburn.so.4 anyway.

---

struct isoburn_read_opts and struct isoburn_source_opts need constructor, destructor and getter, setter methods. They will become opaque handles with no chance for the app to allocate them locally.

(So getter, setter has importance for API safety, despite my earlier

statements via mail.)

---

Unimplemented call isoburn_perform_write() will be disabled in libisoburn.h until somebody implements it.

[vreixo]

Revision 299 now has version 0.6.1. I don't understand the utility isoburn_is_compatible(). I think SONAME is enought for that. Any version (Major.Minor.Micro) will be compatible with a later version unless SONAME is changed.

[scdbackup]

Compatible with future versions, not with past ones.

What about the non-compatibility with older library versions of the same SONAME ?

Example:

xorriso gets built with headers of libisofs-0.6.4, SONAME 6, library file libisofs.so.6.3.0.

User installs libburn libisofs.so.6.1.0 which implements the API evolution of libisofs-0.6.2. xorriso must recognize that this is based on an evolution step which is too old. It has either to restrict itself to that older feature set (a subset of 0.6.4) or it has to refuse to start working.

cdrskin and xorriso will simply refuse.

[scdbackup]

Ignore the word "libburn" before "libisofs.so.6.1.0"

[vreixo]

We have iso_lib_version() for that. The app (xorriso) must known its minimum requirements (libisofs-0.6.4). If the version of the installed library is older (libisofs-0.6.2), the program just can't start (if so, thee requeriments would be 0.6.2, not 0.6.4). If the installed version is newer (0.6.5, for example), the program can safety start unless SONAME was different.

If what you suggest is to program this check in a library function, ok, we can do it, but it is very simple...

[scdbackup]

For the interpretation of _version() we need a rule which is then part of the API and may only be altered in a compatible way. Tricky.

If we have an official compatibility check then we are free to change the numbering scheme without breaking API semantics.

In worst case such a function can maintain a database of historic releases and individually decide whether the header files of a.b.c and the dynamic API of x.y.z do match or not.

[scdbackup]

This how libisoburn allows its applications to ensure runtime compatibility. The build time numbers are defined in libisoburn.h and known to the app. They may be compared against the runtime numbers which stem from libisoburn.h at the build time of the dynamic library.

I propose to provide a similar scheme in libisofs. The test gesture for the app can hardly be more simple.

/** Check whether all features of header file libisoburn.h from the given
    major.minor.micro revision triple can be delivered by the library version
    which is performing this call.
    An application of libisoburn can easily memorize the version of the
    libisofs.h header in its own code. Immediately after isoburn_initialize()
    it should simply do this check:
        if (! isoburn_is_compatible(isoburn_header_version_major,
                                    isoburn_header_version_minor,
                                    isoburn_header_version_micro, 0))
           ...refuse to start the program with this dynamic library version...
    @param major obtained at build time
    @param minor obtained at build time
    @param micro obtained at build time
    @param flag Bitfield for control purposes. Unused yet. Submit 0.
    @return 1= library can work for caller
            0= library is not usable in some aspects. Caller must restrict
               itself to an earlier API version or must not use this libray
               at all.
*/
int isoburn_is_compatible(int major, int minor, int micro, int flag);


/** These three release version numbers tell the revision of this header file
    and of the API it describes. They are memorized by applications at build
    time.
*/
#define isoburn_header_version_major  0
#define isoburn_header_version_minor  0
#define isoburn_header_version_micro  1
/** Note:
    Above version numbers are also recorded in configure.ac because libtool
    wants them as parameters at build time.
    For the library compatibility check ISOBURN_*_VERSION in configure.ac
    are not decisive. Only the three numbers above do matter.
*/


/** Obtain the three release version numbers of the library. These are the
    numbers encountered by the application when linking with lisbisoburn,
    i.e. possibly not before run time.
    Better do not base the fundamental compatibility decision of an application
    on these numbers. For a reliable check use isoburn_is_compatible().
    @param major The maturity version (0 for now, as we are still learning)
    @param minor The development goal version.
    @param micro The development step version. This has an additional meaning:

                 Pare numbers indicate a version with frozen API. I.e. you can
                 rely on the same set of features to be present in all
                 published releases with that major.minor.micro combination.
                 Odd numbers indicate that API upgrades are in progress.
                 I.e. new features might be already present or they might
                 be still missing.
                 So micro revisions {1,3,5,7,9} should never be used for
                 dynamic linking unless the proper library match can be
                 guaranteed by external circumstances.
*/
void isoburn_version(int *major, int *minor, int *micro);

[scdbackup]

I declare the libisoburn API ready for SONAME 1 depending on libisofs SONAME 6.

It is expected that SONAME 1 can be kept up for quite a while. The only possible reason i can imagine now would be a SONAME change of libisofs. It would be too risky to keep the libisoburn SONAME in that case.

---

The only obstacle i see for libisofs to become ready for SONAME 6 is the lack of public header version macros like above in libisoburn:

/** These three release version numbers tell the revision of this header file
    and of the API it describes. They are memorized by applications at build
    time.
*/
#define isoburn_header_version_major  0
#define isoburn_header_version_minor  0
#define isoburn_header_version_micro  1

If they would belong to the initial feature set of libisofs.so.6 then i see no more obstacle to officially open our libraries to adventurous app programmers.

---

(We still have to solve the issue of public libisofs development version download. But that is not topic of this ticket.)

[vreixo]

I don't like the macros idea. Reason: it is the developer who knows what libisofs version its app needs. (S)he is responsible for hardcode the needed numbers in its code. However, with your macro idea in the code not version numbers appear, just macros. So the decission is now made by the compiler and will depend on the vesion installed on the system where the lib is compiled.

For example.

I've written an app. I know my app works with libisofs 0.6.2. However, I use the macros instead of hardcoding 0.6.2. An user downloads my app tarball and compiles it in his system, where compatible libisofs 0.6.9 is installed. Automatically, my app now artificially depends on a later libisofs version, even if I don't use new features from 0.6.2. Now a serious bug in 0.6.9 is found. The user can't wait for a bug fix, and reverts to stable 0.6.8. Automatically the apps refuse to start, and he needs to compile it again. Ugly. That's not the purpose of a shared libray.

And there's even a worst situation. Suppose we have added a meaning to the flag value of a given function in 0.6.8. No more changes from 0.6.7. Our app relies in that parameter. However, the user has 0.6.7. When it compiles (it compiles, as the compiler doesn't know how to check valid parameter values), with your macros idea the dependence of our app falls down to 0.6.7. But it won't work later, as the value we use for flag is not defined in the installed lib.

Conclusion: macros in header are a bad solution. I propose not to include them in any header.

[scdbackup]

This is not the conflict case which is the initial topic of this ticket. It began about runtime confusion. The new examples are about compile time confusion.

To avoid compile time confusion you have to compare the app's expectations with the header revision and both with the revisions of the installed libraries at compile time and at run time.

Up to now this ticket assumed that at compile time everything was of matching revisions. It rather tries to avoid miscombinations which happen after compile time by copying of binaries or installing of system wide libraries.

Vreix: And there's even a worst situation.

This is incorrect. The app compiled with header 0.6.9 would refuse to start with library 0.6.7 if the app programmer followed my advise to run the _is_compatible check as first.

If the app would test its own idea of revisions, then it would start with libburn-0.6.7. But even then it would only use features which have been kept compatible since 0.6.2.

Why else would we invest so much work in changes and then promise to observe compatibility discipline ?

---

The macros cannot do harm because nobody is forced to use them

They would be needed for an eventual check against compile time confusion.

[vreixo]

This is incorrect. The app compiled with header 0.6.9 would refuse to start with library 0.6.7 if the app programmer followed my advise to run the _is_compatible check as first.

yes, and that is wrong. If the app works with 0.6.7, we must be able to use it with 0.6.7, despite of the header we are using. Your solution doesn't allow that!!

[scdbackup]

There is no must to allow a non monotonous version chain of app, header, runtime library. I deem it rather unwise to allow the runtime library to be older than the compiletime header.

Whatever, we will express this difference of views in the API docs.

I got a proposal for the compile time check of app - header compatibility, meanwhile. It uses both, the app's idea of minimum version requirement and the header's version macros:

/* This is the minimum requirement of cdrskin towards the libburn header
   at compile time.
     burn_header_version_major
     burn_header_version_minor  
     burn_header_version_micro
   If the header is too old then the following code shall cause failure of 
   cdrskin compilation rather than to allow production of a program with 
   unpredictable bugs or memory corruption.
   The compiler message supposed to appear in this case is:
      error: 'INTENTIONAL_ABORT_OF_COMPILATION__HEADERFILE_libburn_dot_h_TOO_OLD__SEE_cdrskin_dot_c' undeclared (first use in this function)
*/
#define MajoR 0
#define MinoR 4
#define MicrO 2

/* The indendation is an advise of man gcc to help old compiler ignoring */
 #if MajoR > burn_header_version_major
 #define Cdrskin_libburn_dot_h_too_olD 1
 #endif
 #if MajoR == burn_header_version_major && MinoR > burn_header_version_minor
 #define Cdrskin_libburn_dot_h_too_olD 1
 #endif
 #if MinoR == burn_header_version_minor && MicrO > burn_header_version_micro
 #define Cdrskin_libburn_dot_h_too_olD 1
 #endif

#ifdef Cdrskin_libburn_dot_h_too_olD
 INTENTIONAL_ABORT_OF_COMPILATION__HEADERFILE_libburn_dot_h_TOO_OLD__SEE_cdrskin_dot_c = 0;
#endif

Now i would need a way to put this into a neat macro. To be used in the app like

 LIBBURN_COMPILETIME_HEADER_CHECK( major, minor, micro)

[vreixo]

I deem it rather unwise to allow the runtime library to be older than the compile time header.

Back to one of my examples. Let's suppose an app developed with libisofs 0.6.2. An user that has libisofs 0.6.9 compiles it. Automatically (due the macro system) the app is now dependent of libisofs 0.6.9. Now a bug is found in 0.6.9, and the user choose to revert to stable 0.6.8. But now, the apps refuse start, because of the macro usage. Without macros it could start, and it works propertly.

Macros no, please! Compilation checks must be done in configure.ac, for example with PKG_CHECK_MODULES.

[scdbackup]

Every second time we worked on configure.ac we shot our own foot.

I prefer to have a check which does not depend on autotools. So my code can detect when i did it wrong in configure.ac.

Am i right that we differ in these two opinions:

1) Whether at runtime to check against the application's idea of required library (Vreixo) or whether to check against the header version from build time (Thomas).

2) Whether at compile time to leave all compatibility checks to the build system (Vreixo) or whether to have own application checks based on app requirements and header version (Thomas)

Both differences are about recommendations towards the app developers and do not affect the API feature set. So we can now have them co-existing in the lib*.h files.

[deewang]

Thanks for a resourceful content. this is what i have been looking for so long. I don't get the idea why they start these kind of trial when the criminal are old. Perhaps this more personal form of accountability will deter future actors where the fear of international condemnation has not. Ed Hardy stores are located in many locations internationally including the Americas, Europe and Asia.

---

 Cheap Ed Hardy |  Ed Hardy Shop |  Ed Hardy Sale |  Ed Hardy UK |  Ed Hardy Clothing |  Ed Hardy London |  Ed Hardy Apparel |  Ed Hardy T-Shirts