Is it possible to write to a file in Go with a specific encoding like windows-1252 for instance?
You will have to use a library to convert between encodings:
https://code.google.com/p/go-charset/
The library allows you to convert strings to and from encodings like windows-1252.
Project go.text provides utilities for this precise case, with a Windows-1252 charmap defined.
Related
I need the Pascal (Delphi or Free Pascal) function to convert UTF-8 text (e.g. in string) to Shift-JIS (also e.g. in string).
For Shift-JIS => UTF-8 I found the code, which is here. Now I need the backward converter.
I want to add this code to Lazarus library LConvEncoding, so it must be without using 'iconv' or Win32 API, it must be pure Pascal. It may be code in another language if it's easy to rewrite it in Pascal.
I found the answer, this code can be made from the Unicode official data:
ftp://ftp.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/JIS/SHIFTJIS.TXT .
That file tells how to get UTF-16 codes, and next I must convert UTF-16 -> UTF-8, using UTF8Encode().
I load csv file of utf-8 encoding with cyrillic strings. After parsing in Flow interface - i see not cyrillic, but not readable symbols like "пїўпѕЂпѕ™пїђпѕ" How can i use utf-8 cyrillic strings in H2O?
This appears to be a bug in the Flow interface, but only in the setupParse command. If you continue through and do the import, the data gets imported correctly.
I've reported the bug, with test data and screenshots (taken in Firefox) here:
https://0xdata.atlassian.net/browse/PUBDEV-4640
So if you have additional information, or the bug is behaving differently for you, it'd be good to add it to that bug report.
check your csv file in text and binary presentation to find how Cyrillic text is encoded, if it is UTF-8 it should look like this:
Привет
for the word
Привет
I implement org.osgi.service.cm.ManagedService interface to get Karaf configuration. But when I give a Chinese value to the property, it is garbled.Initially, the files in the etc folder are encoded in latin1. I have tried to set utf-8 encoding, but it has no effect. Can anyone help me?
In Karaf, the configurations files (ie etc/*.cfg) are handled by the felix subproject "fileinstall".
fileinstall doesn't support yet to specified a custom character encoding for the configuration, it uses the Properties class and Properties.load(InputStream), which documents:
The load(Reader) / store(Writer, String) methods load and store
properties from and to a character based stream in a simple
line-oriented format specified below. The load(InputStream) /
store(OutputStream, String) methods work the same way as the
load(Reader)/store(Writer, String) pair, except the input/output
stream is encoded in ISO 8859-1 character encoding. Characters that
cannot be directly represented in this encoding can be written using
Unicode escapes as defined in section 3.3 of The Java™ Language
Specification; only a single 'u' character is allowed in an escape
sequence. The native2ascii tool can be used to convert property files
to and from other character encodings.
So, you have to encode your file in ISE-8859-1 and quote every UTF character, or use an xml file to encode your configuration files.
There is a way to change cfg files encoding.
Configuration for fileinstall subproject polling etc/*.cfg files is written in config.properties file.
You can add
felix.fileinstall.configEncoding = UTF-8
The solution was checked in Karaf 4
I have used Jekyll tool to convert MarkDown file To HTML. It has been successfully converted to HTML. but the below following encoded punctuation characters has been added at the top of the HTML, due to the file encoded format is Encode in UTF-8.
"—-"
After changed the same markdown file to Encode in ANSI format in NotePad++[encoding option in menu bar]. The punctuation character not included in generated HTML.
In this we need to manually change the markdown file to ANSI for HTML generation 'Jekyll'.
Any solution for this?
 is the UTF-8 BOM so that's probably what you are seeing, assuming you are looking at it using CP1252; and — is something out of the General Punctuation block.
Proper diagnostics are not possible without an indication of which character encoding you are using instead of UTF-8 to view the file, and/or what exact bytes you have in the file, probably as a hex dump. The first few bytes (the BOM) would be EF BB BF. See also the character-encoding tag wiki for troubleshooting tips.
Quick googling indicates that Jekyll is highly allergic to UTF-8 BOM in its input, so it seems unlikely that it generates spurious BOM characters on output. I could speculate that the template file you are using has a BOM and that it is being faithfully included in the output, but I'm not really familiar enough with Jekyll to actually help troubleshoot any further.
Of course, as per the big ugly warnings all over the Jekyll site, I assume you have already made sure that your Markdown input doesn't have a BOM character. Many Windows editors are notorious for putting one in when you save as UTF-8; make sure you use "UTF-8 without a BOM" as the "Save As..." format -- and switch to an editor which offers this option if yours doesn't have it.
try using charset=utf-8
or
Check your content has any straight double quote (" ") or straight single quote (' ') and remove those
http://practicaltypography.com/straight-and-curly-quotes.html
This encoding format issue. make the markdown file in UTF-8 without BOM format.
This will remove the punctuation character in 'html' .
What the Windows 'hosts' file encoding is? Is it UTF-8? Or ASCII + system codepage? How IDN (international domain names with umlauts etc.) entries should be added and can they be added at all?
It should be ANSI or UTF-8 without BOM. I just dealt with a server that had the hosts file encoding set to UCS-2 Little Endian, and that led to the file being ignored.
There is a wealth of information here:
https://serverfault.com/questions/452268/hosts-file-ignored-how-to-troubleshoot
The simple answer is
ANSI or UTF-8 WITH BOM.
(UTF-8 without BOM is NOT valid).
Details:
As far as I have tried, the encoding of the hosts file on Windows should be
ANSI or UTF-8 with BOM.
I know this question is many years old, but a colleague made the mistake of looking at this post and the ServerFault post, so I decided to add an answer.
1. Simple case only ASCII
Works.
Without any multi-byte characters, This is equivalent to ANSI, also equivalent to UTF-8 without BOM.
2. ANSI (with Japanese ANSI multi-byte characters)
Works.
note: There are Japanese characters but this is valid ANSI encoding in windows.
In Japanese editions of Windows, this code page cp932 is referred to as "ANSI",
https://en.wikipedia.org/wiki/Code_page_932_(Microsoft_Windows)
3. UTF-8 with BOM
Works.
note: BOM 付き means with BOM.
4. UTF-8 without BOM
DOES NOT work.
5. Additional test cases
If you use emoji instead of Japanese, the result will be the same.
Use emoji and save as UTF8 without BOM does not work.
(However, other lines not include emoji may be worked correctly.)
Use emoji and save as UTF8 with BOM can resolve host correctly.
note: If you use Notepad to check it yourself, be sure to put double quotes in the file name when you save it, or Notepad will be create hosts.txt.
Appended:
(Asked in comment)
The hosts file supports inline comments.