Comments on: Writing UTF-8 files in C++

By: hamed ahmad

hamed ahmad — Mon, 09 Apr 2012 11:54:14 +0000

Thanks A Greate example I needed
I got some problems with appendig to wstring but there is allways a work around

By: Ben Staniford

Ben Staniford — Tue, 27 Mar 2012 16:24:32 +0000

Hmm, there’s two things I don’t like about this:

1. Casting away a const and then writing to the underlying buffer is undefined behavior. As of C++11 you could use &newbuffer[0] which gives you a non-const pointer and is designed for this purpose.

2. Storing utf-8 in the buffer of a string is surely asking for trouble… What if someone later attemps to use one of strings algorithms on the buffer? Will it work? Maybe if the utf-8 characters all happen to be narrow, probably not if there are any wide ones in there. This is also basically undefined behvior and the only thing you could safely do with that std::string is immediately write it out.

By: william

william — Mon, 16 Jan 2012 18:37:11 +0000

would be nice to see a linux compatible article

By: quandaso

quandaso — Thu, 15 Dec 2011 13:49:36 +0000

Thanks, very useful!

By: Getting started with Unicode « Big O(random)

Getting started with Unicode « Big O(random) — Tue, 02 Aug 2011 16:17:31 +0000

[...] set up some form of automatic conversion that hooks into the C++ streams library. For example, see Writing UTF-8 files in C++ by Marius Bancila. This is information I’m going to keep in mind, but my testing with GCC 4.5 [...]

By: sam

sam — Fri, 22 Jul 2011 17:44:01 +0000

Hi there,

thanks for the code it’s exactly I was looking for.
Could you just help me out with one more thing? How can I read utf-8 files?

By: Marius Bancila

Marius Bancila — Sun, 15 Aug 2010 20:28:32 +0000

@shahab, sorry this is not a forum where you can ask questions like this. I suggest to bring this problem in a forum like http://www.codeguru.com/forum.

By: shahab

shahab — Sun, 15 Aug 2010 16:59:50 +0000

hi,i need a help.pleasssssssse
i want to write a text like:

“i am a student.i’m studing computer science at school.i love programming.i want to be very good in it.”

in a text file.dat or .txt and then read it in c++
in order to find out how many time, say computer word, repaeat in the text
?hjow?

By: Silviu

Silviu — Fri, 22 Jan 2010 07:53:19 +0000

Sorin, if you’re using _wfopen() with css=UTF-8, you automatically add BOM chars to the file beginning. The BOM should create more disadvantages then advantages.
For instance, if you open again the same file created with the same parameters (just to change your file content) then you must be careful to remove the old BOM chars section, because otherwise you have two BOMs.
I met this situation in the past into a bug of one of my colleges and it isn’t to pretty.
I use std::ofstream class and a class conversion that contains a similarly conversion method, Marius’s sample.

By: Collation, DateTime, SParse Column and XML : Unicode References

Collation, DateTime, SParse Column and XML : Unicode References — Wed, 20 May 2009 05:01:40 +0000

[...] about UTF-8 Encoding. Then he gave a C++ code example of convert from/to UTF-16 to UTF-8. This is another example of writing UTF-8 in [...]