C++ unicode latin symbol replace? - c++11

Question

Starting from a text that contains characters like \u00f9, \u00a0, \u00e8 I would like to replace them with the ascii equivalents ù, è, etc.
There is my current implementation, that for some reason every now and then it delete pieces of other words and I don't understand why:
pos1 = str2.find("\\u00a0");
pos2 = str2.find("\\u00");
pos3 = str2.find("\\u20");
pos4 = str2.find("\\r\\n");
while (pos1 != std::string::npos)
{
str2.replace(pos1, 6, "");
pos1 = str2.find("\\u00a0");
}
while (pos2 != std::string::npos)
{
str2.replace(pos2, 6, "?");
pos2 = str2.find("\\u00");
}
while (pos3 != std::string::npos)
{
str2.replace(pos3, 6, "?");
pos3 = str2.find("\\u20");
}
while (pos4 != std::string::npos)
{
str2.replace(pos4, 2, "\n");
pos4 = str2.find("\\r\\n");
}
and there's an example of the text:
William Shakespeare \u00e8 stato un drammaturgo e poeta inglese, considerato come il pi\u00f9 importante scrittore in inglese e generalmente ritenuto il pi\u00f9 eminente drammaturgo della cultura occidentale.\u00a0\r\n

Answer 1

One problem I see is that \u... character sequences are always 6 characters, but you are only looking for and replacing 4 characters leaving the other 2 characters in the string.
Also, you are searching for all of the sequences up front, but then you are not taking into account that any replacements the pos1 loop makes will invalidate pos2, pos3 and pos4. Then any replacements the pos2 loop makes will invalidate pos3 and pos4. And so on.
I would suggest a completely different parsing strategy:
#include <string>
#include <sstream>
#include <iomanip>
...
auto pos = str2.find('\\');
char ch;
while (pos != std::string::npos)
{
if (++pos == str2.size())
break;
switch (str2[pos])
{
case 'r':
ch = '\r';
str2.replace(pos-1, 2, &ch, 1);
/* or:
str2.replace(--pos, 2, "");
*/
break;
case 'n':
ch = '\n';
str2.replace(pos-1, 2, &ch, 1);
break;
case 't':
ch = '\t';
str2.replace(pos-1, 2, &ch, 1);
break;
case '\\':
ch = '\\';
str2.replace(pos-1, 2, &ch, 1);
break;
case 'u': {
std::istringstream iss(str2.substr(pos+1, 4));
unsigned short u;
if (!(iss >> std::hex >> u)) {
ch = '?';
}
else
{
switch (u)
{
case 0x2019:
ch = '\'';
break;
...
default:
if (u <= 256)
ch = static_cast<char>(u);
else
ch = '?';
break;
}
}
str2.replace(pos-1, 6, &ch, 1);
break;
}
}
pos = str2.find('\\', pos);
}
Live Demo