{"id":267,"date":"2016-05-26T11:45:45","date_gmt":"2016-05-26T03:45:45","guid":{"rendered":"http:\/\/www.cclo.idv.tw\/blogs\/closer\/?p=267"},"modified":"2021-11-12T19:57:48","modified_gmt":"2021-11-12T11:57:48","slug":"utf-8-%e8%88%87-bom","status":"publish","type":"post","link":"https:\/\/blog.cclo.idv.tw\/?p=267","title":{"rendered":"UTF-8 \u8207 BOM"},"content":{"rendered":"\n

Unicode \u7c21\u4ecb<\/h2>\n\n\n\n

Unicode\uff08\u4e2d\u6587\u7ffb\u6210\u300c\u842c\u570b\u78bc\u300d\u6216\u300c\u7d71\u4e00\u78bc\u300d\uff09\u662f\u4e00\u7a2e\u6587\u5b57\u7de8\u78bc\u65b9\u5f0f\u3002\u4e3b\u8981\u7684\u76ee\u7684\uff0c\u662f\u5e0c\u671b\u80fd\u63d0\u4f9b\u4e00\u5957\u7d71\u4e00\u7684\u6a19\u6e96\uff0c\u4ee5\u6578\u5b57\u7684\u65b9\u5f0f\u8868\u793a\u5168\u4e16\u754c\u4eba\u985e\u4f7f\u7528\u7684\u6587\u5b57\uff0c\u65b9\u4fbf\u5229\u7528\u96fb\u8166\u3001\u7db2\u8def\u5132\u5b58\u53ca\u50b3\u8f38\u6587\u5b57\u8cc7\u8a0a\u3002\u8a73\u7d30\u7684\u6b77\u53f2\u6cbf\u9769\uff0c\u7db2\u8def\u4e0a\u53ef\u4ee5\u627e\u5230\u5f88\u591a\u6587\u7ae0\uff0c\u5728\u6b64\u4e0d\u518d\u8d05\u8ff0\u3002<\/p>\n\n\n\n

\u95dc\u65bc\u78bc\u9ede Code Point<\/h2>\n\n\n\n

\u6bcf\u4e00\u500b\u88ab Unicode \u898f\u7bc4\u5b9a\u7fa9\u7684\u6587\u5b57\uff0c\u90fd\u6709\u4e00\u500b\u7368\u4e00\u7121\u4e8c\u7684\u6578\u503c\uff0c\u6211\u5011\u7a31\u4e4b\u70ba\u300c\u78bc\u9ede<\/strong> (Code Point\uff0c\u4ea6\u7ffb\u4f5c\u300c\u4ee3\u78bc\u9ede\u300d)\u300d\u3002\u901a\u5e38\u6211\u5011\u6703\u7528 U+xxxx<\/code>\uff08xxxx<\/code> \u4ee3\u8868 16 \u9032\u4f4d\u7684\u6578\u5b57\uff09\u9019\u6a23\u7684\u683c\u5f0f\u4f86\u8868\u793a\u78bc\u9ede\u3002<\/p>\n\n\n\n\n\n\n\n

\u8209\u4f8b\u4f86\u8aaa\uff0c\u4e2d\u6587\u7684\u300c\u6211\u300d\u9019\u500b\u5b57\u7684\u78bc\u9ede\u662f U+6211<\/code>\uff0c\u4ee5 16 \u9032\u4f4d\u4f86\u8868\u793a\u7684\u8a71\u662f 0x6211<\/code>\uff0c\u63db\u7b97\u6210 10 \u9032\u4f4d\u5c31\u662f 25105\u3002\u4e0b\u9762\u9019\u6bb5\u7c21\u55ae\u7684 HTML \u6587\u4ef6\uff1a<\/p>\n\n\n

  <BODY<\/span>><\/span>\n    <H1<\/span>><\/span>&#25105<\/H1<\/span>><\/span>\n    <H2<\/span>><\/span>&#x6211<\/H2<\/span>><\/span>\n  <\/BODY<\/span>><\/span><\/code><\/span>Code language:<\/span> HTML, XML<\/span> (<\/span>xml<\/span>)<\/span><\/small><\/pre>\n\n\n
\u6703\u986f\u793a\u5927\u5c0f\u4e0d\u540c\u7684\u5169\u500b\u300c\u6211\u300d\u5b57\u3002<\/p>\n\n\n\n
\u7de8\u78bc\u65b9\u5f0f<\/h2>\n\n\n\n
\u96d6\u7136\u6bcf\u500b\u6587\u5b57\u90fd\u6709\u4e00\u500b\u78bc\u9ede\uff0c\u4f46\u5be6\u969b\u61c9\u7528\u4e0a\uff0c\u70ba\u4e86\u96fb\u8166\u5132\u5b58\u3001\u8655\u7406\u3001\u50b3\u8f38\u65b9\u4fbf\uff0c\u4eba\u5011\u767c\u5c55\u51fa\u4e86\u5404\u7a2e\u4e0d\u540c\u7684\u7de8\u78bc\u65b9\u5f0f\u3002\u901a\u5e38\u6211\u5011\u7a31\u4e4b\u70ba Unicode Transformation Format\uff0c\u7e2e\u5beb\u70ba UTF<\/strong>\uff0c\u4e2d\u6587\u8b6f\u70ba\u300cUnicode \u8f49\u63db\u683c\u5f0f<\/strong>\u300d\u3002\u5e38\u898b\u7684\u7de8\u78bc\u65b9\u5f0f\u6709 UTF-7\u3001UTF-8\u3001UTF-16 \u4ee5\u53ca UTF-32 \u7b49\u3002\u672c\u6587\u4e3b\u8981\u8a0e\u8ad6\u7684\u70ba UTF-8 \u8207 UTF-16\u3002<\/p>\n\n\n\n
UTF-16<\/h3>\n\n\n\n
UTF-16 \u662f Windows \u548c Mac OS X \u9810\u8a2d\u7684 Unicode \u7de8\u78bc\u65b9\u5f0f\u3002\u540d\u7a31\u4e2d \"16\" \u7684\u610f\u7fa9\u662f\u300c\u4ee5 16 \u4f4d\u5143 (bit) \u7684\u6578\u5b57\u70ba\u55ae\u4f4d\uff0c\u53bb\u8868\u793a Unicode \u7684\u6587\u5b57\u300d\u3002\u78bc\u9ede\u5728 U+FFFF<\/code> \u4ee5\u4e0b\u7684\u5b57\uff0c\u5176 UTF-16 \u7de8\u78bc\u57fa\u672c\u4e0a\u5c31\u8ddf\u78bc\u9ede\u4e00\u6a23\u3002\u4f8b\u5982\u300c\u6211\u300d\u7684 UTF-16 \u7de8\u78bc\u5c31\u662f 0x6211<\/code>\u3002<\/p>\n\n\n\n
UTF-8<\/h3>\n\n\n\n
UTF-16 \u6709\u4e00\u500b\u7f3a\u9ede\uff1a\u5728 Unicode \u767c\u660e\u4ee5\u524d\uff0c\u5927\u90e8\u4efd\u7684\u6587\u5b57\u6a94\u6848\u90fd\u662f\u4ee5 ASCII \u683c\u5f0f\u5132\u5b58\u7684\uff1b\u4f46 UTF-16 \u548c ASCII \u4e26\u4e0d\u76f8\u5bb9\u3002\u8209\u4f8b\u4f86\u8aaa\uff0cABC<\/code> \u9019\u500b\u5b57\u4e32\u4ee5 ASCII \u8868\u793a\u7684\u8a71\uff0c\u5c31\u662f 0x41 0x42 0x43<\/code>\uff0c\u4f46 UTF-16 \u5247\u70ba 0x00 0x41 0x00 0x42 0x00 0x43<\/code>\u3002\u9019\u500b\u554f\u984c\u6703\u5c0e\u81f4\u73fe\u6709\u8655\u7406\u6587\u5b57\u7684\u7a0b\u5f0f\u3001\u51fd\u5f0f\u5eab\u5168\u90e8\u5f97\u8981\u91cd\u5beb\uff0c\u73fe\u6709\u7684\u6587\u5b57\u6a94\u6848\u4e5f\u5168\u90e8\u5f97\u8981\u8f49\u63db\u6210 UTF-16 \u624d\u80fd\u4f7f\u7528\u3002<\/p>\n\n\n\n
\u70ba\u4e86\u89e3\u6c7a\u9019\u500b\u554f\u984c\uff0c\u6709\u4eba\u767c\u660e\u4e86 UTF-8 \u9019\u7a2e\u683c\u5f0f\u3002\u539f\u672c ASCII \u5b9a\u7fa9\u7684\u7bc4\u570d\uff08\u78bc\u9ede U+0000<\/code> ~ U+007F<\/code>\uff09\u7684\u6587\u5b57\uff0cUTF-8 \u548c ASCII \u5b8c\u5168\u76f8\u5bb9\u3002\u4f46\u6709\u4e00\u597d\u6c92\u5169\u597d\uff1a\u78bc\u9ede\u7a0d\u5fae\u5927\u4e00\u9ede\uff0c\u6240\u9700\u8981\u7684\u5132\u5b58\u7a7a\u9593\u5c31\u6703\u6210\u9577\u5f97\u5f88\u5feb\u3002\u4f8b\u5982\u300c\u6211\u300d\u7528 UTF-16 \u7de8\u78bc\u7684\u8a71\u53ea\u9700\u8981 2 \u500b\u4f4d\u5143\u7d44 (byte) \u7684\u7a7a\u9593\uff1b\u4f46\u7528 UTF-8 \u5247\u9700\u8981 3 \u500b\u4f4d\u5143\u7d44 (0xE6 0x88 0x91<\/code>)\u3002\u81f3\u65bc\u78bc\u9ede\u5982\u4f55\u8f49\u63db\u6210 UTF-8 \u7684\u7de8\u78bc\uff0c\u8acb\u81ea\u884c google\u3002<\/p>\n\n\n\n
Linux \u9810\u8a2d\u7684 Unicode \u7de8\u78bc\u65b9\u5f0f\u5c31\u662f UTF-8\u3002\u6b64\u5916\uff0c\u76ee\u524d\u4e16\u754c\u4e0a\u7d55\u5927\u90e8\u4efd\u7684\u7db2\u9801\u4e5f\u662f\u4ee5 UTF-8 \u65b9\u5f0f\u5132\u5b58\/\u50b3\u8f38\u7684\u3002<\/p>\n\n\n\n
\u5728\u6b64\u6574\u7406\u4e00\u4e0b\u5e7e\u500b\u6587\u5b57\u7684\u78bc\u9ede\u8207\u7de8\u78bc\u6bd4\u8f03\u8868\uff1a<\/p>\n\n\n\n
 \u6587\u5b57 <\/td> \u78bc\u9ede <\/td> UTF-16 <\/td> UTF-8 <\/td><\/tr>
A<\/td>U+0041<\/code><\/td>0x0041<\/code><\/td>0x41<\/code><\/td><\/tr>
\u00a3<\/td>U+00A3<\/code><\/td>0x00A3<\/code><\/td>0xC2 0xA3<\/code><\/td><\/tr>
\u6211<\/td>U+6211<\/code><\/td>0x6211<\/code><\/td>0xE6 0x88 0x91<\/code><\/td><\/tr>
\ud800\udf3b<\/td>U+1033B<\/code><\/td>0xD800 0xDF3B<\/code><\/td>0xF0 0x90 0x8C 0xBB<\/code><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\u4f4d\u5143\u9806\u5e8f\u8207 BOM<\/h2>\n\n\n\n
UTF-16 \u9084\u6709\u4e00\u500b\u554f\u984c\u3002\u5b83\u7684\u7de8\u78bc\u57fa\u672c\u55ae\u4f4d\u70ba 16 \u4f4d\u5143\u7684\u6578\u5b57\uff1b\u4f46\u6587\u5b57\u6a94\u6848\u5132\u5b58\u6642\u5927\u591a\u662f\u4ee5\u4f4d\u5143\u7d44\uff08\u9577\u5ea6\u70ba 8 \u4f4d\u5143\uff09\u70ba\u55ae\u4f4d\u3002\u7576\u6211\u5011\u60f3\u628a 0x6211<\/code> \u9019\u500b\u5b57\u5b58\u9032\u6a94\u6848\u88e1\u6642\uff0c\u7a76\u7adf\u8a72\u5b58\u6210 0x62 0x11<\/code> \u9084\u662f 0x11 0x62<\/code> \u5462\uff1f<\/p>\n\n\n\n
\u5538\u8cc7\u8a0a\u7684\u4eba\u61c9\u8a72\u90fd\u77e5\u9053\uff0c\u9019\u662f\u5178\u578b\u7684 \"Big-Endian vs Little-Endian\" \u7684\u554f\u984c\u3002\u800c\u9019\u5169\u7a2e\u505a\u6cd5\u5176\u5be6\u4e26\u7121\u512a\u52a3\u4e4b\u5225\uff0c\u56e0\u6b64\u5169\u7a2e\u65b9\u6cd5\u90fd\u7b26\u5408 UTF-16 \u7684\u898f\u7bc4\u3002<\/p>\n\n\n\n
\u4f46\u554f\u984c\u9084\u662f\u5f97\u89e3\u6c7a\u3002\u8b80\u6a94\u6848\u7684\u4eba\uff08\u6216\u7a0b\u5f0f\uff09\u8981\u600e\u9ebc\u77e5\u9053\u9019\u500b\u6a94\u6848\u662f Big-Endian \u9084\u662f Little-Endian \u5462\uff1fUnicode \u7684\u505a\u6cd5\u662f\uff1a\u6240\u6709 UTF-16 \u7684\u6a94\u6848\u958b\u982d\uff0c\u90fd\u6703\u63d2\u5165 U+FEFF<\/code> \u9019\u500b\u300c\u6587\u5b57\u300d\u3002\u8b80\u53d6\u6a94\u6848\u7684\u6642\u5019\uff0c\u85c9\u7531\u89c0\u5bdf\u958b\u982d 2 \u500b\u4f4d\u5143\u7684\u9806\u5e8f\uff080xFE 0xFF<\/code> \u6216\u662f 0xFF 0xFE<\/code>\uff09\uff0c\u5c31\u80fd\u77e5\u9053\u9019\u500b\u6a94\u6848\u4f7f\u7528\u7684\u9806\u5e8f\u70ba\u4f55\u3002<\/p>\n\n\n\n
U+FEFF<\/code> \u9019\u500b\u300c\u6587\u5b57\u300d\u88ab\u7a31\u505a Byte Order Mark\uff0c\u7e2e\u5beb\u70ba BOM<\/strong>\uff0c\u4e2d\u8b6f\u70ba\u300c\u4f4d\u5143\u9806\u5e8f\u8a18\u865f<\/strong>\u300d\u3002<\/p>\n\n\n\n
UTF-8 \u9700\u8981 BOM \u55ce\uff1f<\/h2>\n\n\n\n
\u76f8\u8f03\u65bc UTF-16\uff0cUTF-8 \u7684\u7de8\u78bc\u57fa\u672c\u55ae\u4f4d\u70ba 8 \u4f4d\u5143\uff0c\u4e5f\u5c31\u662f 1 \u500b\u4f4d\u5143\u7d44\uff0c\u56e0\u6b64\u6c92\u6709 endian \u7684\u554f\u984c\u3002\u96d6\u7136 BOM \u80fd\u4ee5 UTF-8 \u8868\u793a\uff08\u7562\u7adf\u5b83\u662f\u6709\u78bc\u9ede\u7684\u5408\u6cd5\u6587\u5b57\uff09\uff0c\u4f46\u5b58\u5728\u7684\u610f\u7fa9\u4e0d\u5927\uff0cUnicode \u7684\u898f\u7bc4\u4e2d\u751a\u81f3\u660e\u6587\u5beb\u8457\uff1a\u300cUTF-8 \u4e0d\u9700\u8981\u4e14\u4e0d\u5efa\u8b70\u4f7f\u7528 BOM\u3002\u300d\u56e0\u70ba\u5b83\u6703\u7834\u58de UTF-8 \u8207 ASCII \u7684\u76f8\u5bb9\u6027\u3002<\/p>\n\n\n\n
BUT!!<\/h2>\n\n\n\n
\u5fae\u8edf\u8aaa\uff1a\u300c(UTF-8) \u8981\u6709 BOM\uff01\u300d\u5c31\u6709\u4e86 BOM....\uff08\u7b11\uff09<\/p>\n\n\n\n
\u5fae\u8edf\u8a8d\u70ba\uff0cUTF-8 \u7684 BOM \u6709\u52a9\u65bc\u7a0b\u5f0f\u5224\u65b7\u8a72\u6a94\u6848\u662f\u5426\u4ee5 UTF-8 \u7de8\u78bc\uff1b\u800c\u4e00\u822c\u5fae\u8edf\u63d0\u4f9b\u7684\u6587\u5b57\u7de8\u8f2f\u5668\uff0c\u4f8b\u5982\u8a18\u4e8b\u672c (Notepad)\uff0c\u5728\u300c\u53e6\u5b58\u65b0\u6a94\u300d\u6642\u82e5\u7de8\u78bc\u9078\u64c7 UTF-8\uff0c\u6a94\u6848\u7684\u524d\u9762\u5c31\u6703\u81ea\u52d5\u88ab\u52a0\u4e0a BOM\u3002<\/p>\n\n\n\n
\u7136\u800c\uff0c\u8a31\u591a\u7a0b\u5f0f\u5728\u8655\u7406 UTF-8 \u7de8\u78bc\u7684\u6a94\u6848\u6642\uff0c\u90fd\u6c92\u6709\u7279\u5225\u8655\u7406 BOM\uff0c\u7d50\u679c\u5c31\u662f\u628a BOM \u7576\u6210\u662f\u591a\u9918\u7684\u5b57\u5143\uff0c\u5c0e\u81f4\u7a0b\u5f0f\u767c\u751f\u932f\u8aa4\u3002<\/p>\n\n\n\n
\u5982\u4f55\u79fb\u9664 UTF-8 \u7684 BOM\uff1f<\/h2>\n\n\n\n
\u5982\u679c\u53ea\u6709\u5c11\u6578\u7684\u5e7e\u500b\u6a94\u6848\u8981\u8655\u7406\uff0c\u53ef\u4ee5\u5229\u7528\u6587\u5b57\u7de8\u8f2f\u5668\u624b\u52d5\u8655\u7406\u3002\u4f8b\u5982\u514d\u8cbb\u7684 Notepad++\uff0c\u5c31\u53ef\u4ee5\u5229\u7528\u9078\u55ae\u4e2d\u7684\u300c\u7de8\u78bc -> \u8f49\u63db\u81f3 UTF-8 \u78bc\u683c\u5f0f\uff08\u6a94\u9996\u7121 BOM\uff09\u300d\u529f\u80fd\uff0c\u5c07\u6a94\u6848\u524d\u9762\u7684 BOM \u62ff\u6389\u3002<\/p>\n\n\n\n
\u5982\u679c\u60f3\u8981\u5927\u91cf\u6279\u6b21\u8f49\u63db\u6a94\u6848\uff0c\u5efa\u8b70\u53ef\u4ee5\u5229\u7528\u6279\u6b21\u6a94\uff08\u6216 shell script\uff09\u642d\u914d\u7c21\u55ae\u7684 awk \u7a0b\u5f0f\u8655\u7406\u3002\u8a73\u7d30\u7684 awk \u7a0b\u5f0f\u53ef\u4ee5\u53c3\u8003\u6587\u7ae0\u5f8c\u7684\u9023\u7d50\u3002<\/p>\n\n\n\n
\n\n\n\n
\u53c3\u8003\u8cc7\u6599<\/h2>\n\n\n\n
  1. Wikipedia: Unicode<\/a> <\/li>
  2. Wikipedia: UTF-8<\/a><\/li>
  3. Usage statistics of character encodings for websites<\/a><\/li>
  4.  UTF-8 Everywhere<\/a> <\/li>
  5.  StackOverflow: How to make Notepad to save text in UTF-8 without BOM?<\/a> <\/li>
  6.  StackOverflow: Using awk to remove the Byte-order mark<\/a> <\/li>
  7.  \u7d04\u8033\u8ac7\u8edf\u9ad4\uff1a\u6bcf\u500b\u8edf\u9ad4\u958b\u767c\u8005\u90fd\u7d55\u5c0d\u4e00\u5b9a\u8981\u6703\u7684Unicode\u53ca\u5b57\u5143\u96c6\u5fc5\u5099\u77e5\u8b58 (\u6c92\u6709\u85c9\u53e3\uff01)<\/a> <\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"
    Unicode \u7c21\u4ecb Unicode\uff08\u4e2d\u6587\u7ffb\u6210\u300c\u842c\u570b\u78bc\u300d\u6216\u300c\u7d71\u4e00\u78bc\u300d\uff09\u662f\u4e00\u7a2e\u6587\u5b57\u7de8\u78bc\u65b9\u5f0f\u3002\u4e3b\u8981\u7684\u76ee\u7684\uff0c\u662f\u5e0c\u671b\u80fd […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[13],"tags":[46,47],"table_tags":[],"class_list":["post-267","post","type-post","status-publish","format-standard","hentry","category--computer-science","tag-unicode","tag-utf-8"],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=\/wp\/v2\/posts\/267","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=267"}],"version-history":[{"count":3,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=\/wp\/v2\/posts\/267\/revisions"}],"predecessor-version":[{"id":567,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=\/wp\/v2\/posts\/267\/revisions\/567"}],"wp:attachment":[{"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=267"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=267"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=267"},{"taxonomy":"table_tags","embeddable":true,"href":"https:\/\/blog.cclo.idv.tw\/index.php?rest_route=%2Fwp%2Fv2%2Ftable_tags&post=267"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}