Ruby 1.8 Iconv UTF-16 for UTF-8 does not work with "\ 000" (Iconv :: InvalidCharacter)

I am having problems processing text files with tabulated data generated on a Windows machine. I work in Ruby 1.8. The following is an error ("\ 000" (Iconv :: InvalidCharacter)) while processing the second line from a file. The first line is converted correctly.

require 'iconv'
conv = Iconv.new("UTF-8//IGNORE","UTF-16")
infile = File.open(tabfile, "r")
while (line = infile.gets)
  line = conv.iconv(line.strip)  # FAILS HERE
  puts line
  # DO MORE STUFF HERE
end

The strange thing is that it reads and converts the first line to a file without any problems. I have a // IGNORE flag in the Iconv constructor - I thought this should have suppressed such an error.

I have been in circles for a while. Any advice would be highly appreciated.

Thank!

EDIT: The hobbs solution fixes this. Thank. Just change the code to:

require 'iconv'
conv = Iconv.new("UTF-8//IGNORE","UTF-16")
infile = File.open(tabfile, "r")
while (line = infile.gets("\x0a\x00"))
  line = conv.iconv(line.strip)  # NO LONGER FAILS HERE
  # DOES MORE STUFF HERE
end

Now I just need to find a way to automatically determine which separator to use.

+3
2

, , , , UTF-16 ( ) . , , gets - UTF-16le, 0x0a 0x00, gets ( strip ) 0x0a.

: ,

ab
cd

UTF-16le.

0x61 0x00 0x62 0x00 0x0a 0x00 0x63 0x00 0x64 0x00 0x0a 0x00
    a         b         \n        c         d         \n

gets 0x0a, strip , 0x61 0x00 0x62 0x00, iconv UTF-8 0x61 0x62 - "ab". gets 0x0a, strip , line 0x00 0x63 0x00 0x64 0x00, - , iconv , , .

/ , , gets "\n" ("\x0a") "\x0a\x00", strip, encoding-clean print puts, ( , ).

Windows, CRLF Windows UTF-16le "\x0d\x00\x0a\x00".

+7

. UTF-8, , .

0