phpBB stack trace/MySQL error when posting non-BMP chars (e.g. emoji)

Not sure if this is known, the couple of related threads I found were strictly about PMs, but this happens everywhere.
I don't think stack traces should be showing on a production server (though that may be a thing with phpBB, I wouldn't be surprised).

I tried submitting a post with an emoji in it, and this happened:
Image
It's not restricted to just emoji, though: any non-BMP (basic multilingual plane) character seems to cause the issue (e.g. some CJK or Indic characters, some arrow symbols etc.).

The cause is most likely that the underlying MySQL DB uses the `utf8` charset instead of `utf8mb4`. This is a typical issue with that kind of setup. Data should probably be migrated to the latter charset at some point. The migration is generally not very difficult, but I'm not familiar with the specifics here at DT, so I can't say for sure. It should be done nonetheless, though (it's 2018 and emoji are really popular).
Last edited by constexpr on 10 Jul 2018, 16:54, edited 1 time in total.
constexpr
User avatar

Unread post10 Jul 2018, 15:15

User avatar
X
constexpr
 
Posts: 18
Joined: 08 Jun 2018, 01:42
Location: Serbia
DT Pro Member: -
 
Blaise170
User avatar
ALPS キーボード

Unread post10 Jul 2018, 15:23

User avatar
X
Blaise170
ALPS キーボード
 
Posts: 1937
Joined: 30 Oct 2014, 22:18
Location: Boston, MA
Main keyboard: Sharp X68000
Main mouse: Logitech G602
Favorite switch: Blue Alps SKCM
DT Pro Member: 0129
 
 
I proxy anything, including keyboards (キーボード / 鍵盤), from both Japan (日本) and China (中國). For more information, you may visit my dedicated webpage here: https://www.keyboards.es/proxying.html

View my current and past keyboards here: wiki/User:Blaise170
Yeah, that's the thread I mentioned in the first sentence. Apart from it only talking about PMs, no progress had been made there, so reviving it didn't really seem like a good idea.
constexpr
User avatar

Unread post10 Jul 2018, 15:42

User avatar
X
constexpr
 
Posts: 18
Joined: 08 Jun 2018, 01:42
Location: Serbia
DT Pro Member: -
 
It's an issue with phpBB itself, not the unicode implementation.
Blaise170
User avatar
ALPS キーボード

Unread post10 Jul 2018, 15:58

User avatar
X
Blaise170
ALPS キーボード
 
Posts: 1937
Joined: 30 Oct 2014, 22:18
Location: Boston, MA
Main keyboard: Sharp X68000
Main mouse: Logitech G602
Favorite switch: Blue Alps SKCM
DT Pro Member: 0129
 
 
I proxy anything, including keyboards (キーボード / 鍵盤), from both Japan (日本) and China (中國). For more information, you may visit my dedicated webpage here: https://www.keyboards.es/proxying.html

View my current and past keyboards here: wiki/User:Blaise170
I'm pretty sure that's not the case, as what failed was an SQL DML statement executed by mysqli (PHP's MySQL API), not a PHP function in and of itself (which would indicate that phpBB was at fault). The error message states that the SQL engine has encountered an invalid string value — exactly what you would expect to see as a result of an encoding issue.

As I mentioned, this is a well known issue with software running MySQL in the back (especially software older than a couple of years), and this is exactly the way you'd expect it to manifest itself on a forum. phpBB may rely on the DB charset being utf8, but I doubt that's the case as the two are separated by an API boundary (mysqli).

Here's further reading on the difference between utf8 and utf8mb4
Here's someone who had the same issue on their phpBB board and got emoji to work by changing the charset to utf8mb4
More on modifying an existing phpBB MySQL DB

But regardless of the charset issue, I feel that PHP stack traces, i.e. debug mode should be disabled on the Deskthority production environment in any case.
constexpr
User avatar

Unread post10 Jul 2018, 16:51

User avatar
X
constexpr
 
Posts: 18
Joined: 08 Jun 2018, 01:42
Location: Serbia
DT Pro Member: -
 

Who is online

Users browsing this forum: No registered users and 7 guests