Re: enum vs int and API/ABI compatibility

[Eric Blake]

On 10/19/2011 05:06 AM, Simon Josefsson wrote:
> This is not related to gnulib strictly, but I'd like to consult the
> wisdom on this list.  I'm considering changing the argument to some
> functions in GNU Libidn from 'enum' to 'int' to silence C++ warnings
> (see [1] for discussion).  The change I'm considering is from:
>
>    typedef enum
>    {
> ...
>    } Idna_rc;
> ...
>    extern IDNAPI const char *idna_strerror (Idna_rc rc);
>
> into
>
>    extern IDNAPI const char *idna_strerror (int rc);
>
> Some things that I wonder:
>
> 1) Is there any platform where this has any implications for the ABI?

Name mangling is part of the ABI, and C++ requires enum names to be part 
of the mangling.  If your functions were labeled extern "C", then there 
is (probably) no ABI difference between enum and int, although I think 
the language standards allow the compiler to use a narrower type than 
int if the enum can be proved to fit in the  narrower type.  But for 
extern C++ functions, changing the argument type means that you are 
linking against a different symbol, so applications compiled against the 
old library will not find the entry point in the new library.

In your code, does IDNAPI expand to "C"?  If so, then mangling is not an 
issue.  If not, then use function overloading to keep the old signature 
while also providing the new signature.

> 1b) Any reasonable platform?

Off hand, I don't know of any compilers that use anything besides int 
for an enum that fits in an int, unless you also had additional compiler 
flags explicitly requesting that enums be packed into the narrowest 
type.  But I don't have enough experience in this area to be confident 
that my answer is definative.

> 2) Are there any other negative risks with the API change?  In C, enum
> and int should be interchangeable, and gcc does not seem to warn against
> mixing these types.  Other compilers?  I'm willing to live with new
> warning messages (which seems unavoidable) but not hard errors.
>
> 2b) For C++?  C++ compilers may warn on code that mixes int and enum,
> but the current approach (with functions returning int's that really are
> enum's) also leads to C++ warnings, so on balance I think the proposed
> change is reasonable as it allows future C++ code to be warning free.
>
> Thanks,
> /Simon
>
> [1] http://thread.gmane.org/gmane.comp.gnu.libidn.general/355

--
Eric Blake   address@hidden    +1-801-349-2682
Libvirt virtualization library http://libvirt.org