Too bad you did not specify the target parameters. So for the following I presume:
CHAR_BIT == 8
sizeof(int) >= 2
- not padding bits
str points to valid and initialised memory which is properly nul-terminated and not larger than INT_MAX (alternatively use size_t to index - thanks @chux).
- and possibly other prerequisites we tend to take for given.
These are typical for most modern implementations.
For sizeof(int) == 2, the initialiser is implementation defined behaviour, because you use an unsigned int constant as initialiser. For wider int this is ok. Things also become more complicated if (int)(0xFACA) + CHAR_MAX > INT_MAX (arithmetically, also only relevant for 2 byte int).
The rest relies on implementation defined behaviour in a more complex way:
1) The addition str[i] + key: Here str[i] is converted to int first, the addition is done as int and yields an int result.
2) The assignment str[i] = ...: Here the int result of the addition is converted to char. For that we have two variants, depending on the signed-ness of char (implementation defined):
- unsigned: The result is converted in a standard defined way to
unsigned char.
- signed: The result is "down-"convert to the "smaller"
signed char in an implementation defined way.
So: no undefined behaviour, but that (and the comments) shows how much you have to keep in mind when using signed integers in C.
But:
There is too much implementation defined behaviour involved and a lot of prerequisites are required (which are quite common, though). Better use unsigned char and unsigned int throughout the code. That will make the code standard compliant and well-behaved. Even for other than CHAR_BIT values. If you rely on 8 bit values, use uint8_t from stdint.h.
charhas no standard signed-ness.int. There is only a conversion tocharfor the assignment.