1

I saw this code somewhere, is this not integer overflow and undefined behaviour?

assume 0 < str[i] < 127

char *msl_decrypt(char *str) {
    char *decrypted;
    unsigned int i;
    int key = 0xFACA;

    for (i = 0; i < strlen(str); ++i) {
        str[i] = str[i] + key; // referring to this line, is it UB?
    }
    ...
16
  • And the answer is: maybe! Commented Feb 11, 2016 at 21:09
  • 2
    @DanielJour: You are wrong. char has no standard signed-ness. Commented Feb 11, 2016 at 21:17
  • 1
    @RPGillespie: Wrong! The calculation is done as int. There is only a conversion to char for the assignment. Commented Feb 11, 2016 at 21:18
  • 1
    I'd say it hardly matters whether it's undefined, because (a) it's silly to be adding a 16-bit key to 8-bit bytes in this way, and much more importantly, (b) the "security" of the encryption involved here is nonexistent. Don't worry whether it's undefined, implementation-defined, or well-defined -- pick a better algorithm, pronto! Commented Feb 11, 2016 at 22:06
  • 1
    @RPGillespie: There are some decades more microcontrollers than Server- + PC + larger ARM -CPUs. Actually every PC has some MCUs: HDD, Mainboard, Keyboard, Mouse, DVD/CD/BD, etc. Even the CPU might have an integrated MCU for e.g. power management. Commented Feb 11, 2016 at 22:41

1 Answer 1

5

Too bad you did not specify the target parameters. So for the following I presume:

  • CHAR_BIT == 8
  • sizeof(int) >= 2
  • not padding bits
  • str points to valid and initialised memory which is properly nul-terminated and not larger than INT_MAX (alternatively use size_t to index - thanks @chux).
  • and possibly other prerequisites we tend to take for given.

These are typical for most modern implementations.

For sizeof(int) == 2, the initialiser is implementation defined behaviour, because you use an unsigned int constant as initialiser. For wider int this is ok. Things also become more complicated if (int)(0xFACA) + CHAR_MAX > INT_MAX (arithmetically, also only relevant for 2 byte int).

The rest relies on implementation defined behaviour in a more complex way:

1) The addition str[i] + key: Here str[i] is converted to int first, the addition is done as int and yields an int result.

2) The assignment str[i] = ...: Here the int result of the addition is converted to char. For that we have two variants, depending on the signed-ness of char (implementation defined):

  • unsigned: The result is converted in a standard defined way to unsigned char.
  • signed: The result is "down-"convert to the "smaller" signed char in an implementation defined way.

So: no undefined behaviour, but that (and the comments) shows how much you have to keep in mind when using signed integers in C.


But:

There is too much implementation defined behaviour involved and a lot of prerequisites are required (which are quite common, though). Better use unsigned char and unsigned int throughout the code. That will make the code standard compliant and well-behaved. Even for other than CHAR_BIT values. If you rely on 8 bit values, use uint8_t from stdint.h.

Sign up to request clarification or add additional context in comments.

6 Comments

Another required assumption would be that int key = 0xFACA; generates an int which is not within 127 of INT_MAX
@M.M: Yes; I also added that (just in some more general form). But please don't ask to remove the prerequisites and make all text general; It's getting later here;-).
if you can slightly more elaborate why you assumed those prerequisites would be nice
@user200312: Because things get even more messed up if e.g. SCHAR_MAX == INT_MAX. The ptr related prerequiste should be clear.
why the padding bits requirement?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.