Ivan's wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
# uchar.h
`wchar_t` seemed like the solution to Unicode, but its encoding is implementation-defined — 16-bit UTF-16 on Windows, 32-bit UTF-32 on Linux — making it useless for writing portable Unicode code. **`uchar.h`** (C11) provides `char16_t` and `char32_t` instead, with specified encodings (UTF-16 and UTF-32 respectively), plus conversion functions to and from multibyte (UTF-8).

```c
#include <uchar.h>

char16_t c16 = u'A';      // UTF-16 code unit (BMP character)
char32_t c32 = U'€';      // UTF-32 code point (always a single unit)

// string literals
const char16_t *s16 = u"hello";   // UTF-16 encoded
const char32_t *s32 = U"hello";   // UTF-32 encoded
```

The conversion functions translate between multibyte (UTF-8) and the fixed-width encodings:

```c
mbstate_t state = {0};
char16_t  c16;
size_t    n;

const char *utf8 = "€";   // 3-byte UTF-8 sequence: E2 82 AC

n = mbrtoc16(&c16, utf8, MB_CUR_MAX, &state);
// n = 3 (bytes consumed), c16 = 0x20AC (the euro sign)

char buf[MB_LEN_MAX];
n = c16rtomb(buf, c16, &state);
// n = 3 (bytes written), buf holds E2 82 AC
```

`mbrtoc32` and `c32rtomb` do the same for UTF-32. All four functions are stateful via `mbstate_t`, which tracks partial multibyte sequences across calls — important when you are feeding a stream of bytes one chunk at a time.

`char16_t` is an alias for `uint_least16_t`. On Windows, where `wchar_t` is 16 bits and UTF-16 is the system encoding, `char16_t` maps directly to `wchar_t`. On Linux, `wchar_t` is 32 bits and the two are entirely distinct types.

## Practice

```c
// compile: gcc -o uchartest uchartest.c
// run: ./uchartest
// description: convert a UTF-8 string to UTF-32 code points and print them

#include <uchar.h>
#include <stdio.h>
#include <string.h>
#include <locale.h>

int main(void) {
    setlocale(LC_ALL, "");

    const char *utf8 = "héllo";  // 'é' is a 2-byte UTF-8 sequence
    mbstate_t   state = {0};
    char32_t    c32;
    size_t      n;

    while (*utf8) {
        n = mbrtoc32(&c32, utf8, strlen(utf8), &state);
        if (n == 0 || n == (size_t)-1 || n == (size_t)-2) break;
        printf("U+%04X (%zu bytes)\n", (unsigned)c32, n);
        utf8 += n;
    }
    return 0;
}
```

You should see `U+0068` through `U+006C` for the ASCII characters and `U+00E9` for `é`, consumed as 2 bytes. This is how you iterate over Unicode code points in a UTF-8 string without a third-party library.