Odd/missing characters in "My Horizontal Life"

[ClaudeM]

I've noticed odd characters appear throughout My Horizontal Life:

locations 1281-86: case of ? CD
locations 2256-63: suite} This
locations 2337-42: while you're pregnant ... ?/ you're
locations 2425-31: I spotted my ? version
locations 2538,46: "Why not}"

There are lots more. Another book I read on my Kindle had trouble with ñ (as in Señor), replacing it with . Is this normal?

I've imported text files (plain .TXT) into my Kindle (just copying them, no conversion) and I've noticed missing characters that are there in the original TXT file on my computer.

I have a Kindle 2 International (Canada) version 2.3.3 (431100003).

[nogdog~6op6ou]

Probably Unicode characters not supported by the Kindle native font, which I believe only supports extended ASCII characters. In other words, the document conversion done by the publisher was done without knowledge (or caring about) the specifics of the Kindle platform. There were a few Unicode font hacks created which might address this (if my guess is right), though with the upcoming 2.5.x firmware release, you might not want to mess with that right now.

[ClaudeM]

Thank you. 

I figured that was the case for the ñ, but for question marks or perhaps entire words (what's supposed to be before "CD", "you're" or "version" in the examples above???)  That's really odd.

It's happened with hyphens too, I was reading Judas Unchained, which I'd purchased for the Kindle even though I had the paperback version. I started noticing several missing hyphens in the Kindle version so I located the text in the paper version and the hyphens were there (like ex-fighter pilot, page 885/87%). Maybe they were non-breaking hyphens that are outside the supported ASCII characters.

Anyhow, thank you. Hopefully the Kindle update will solve that problem. I hope they offer it for download, I worry that being in Canada, it won't show up here for a long time.

[nogdog~6op6ou]

It could also be an issue with optically scanned pages which were not sufficiently proofread, perhaps.

[ClaudeM]

Perhaps... Isn't that more likely with older books, though?

[nogdog~6op6ou]

Perhaps... Isn't that more likely with older books, though?

Just brainstorming a bit.

It could also be that the source manuscript was saved in whatever word processor was used with a Unicode encoding and then directly dumped into whatever conversion method they used to create the e-book file without doing anything to convert non-compatible multi-byte characters.

Isn't speculation without any facts fun? 

[ClaudeM]

Oh, absolutely! Speculating helps develop a critical mind and makes problem-solving easier in other areas.

I've installed Kindle for PC and opened My Horizontal Life. The same odd characters are displayed in the same spots, same missing hyphen in Judas Unchained (thanks for not including a text search function in the PC version!!!) so I'm guessing it's not a Kindle (the device) problem. The files I have are like that and contain those errors.

I've uploaded three of the screen caps I made:

http://pages.infinit.net/ewan/screen_shot-50435.gif (error just above center)
http://pages.infinit.net/ewan/screen_shot-50437.gif (error at bottom right)
http://pages.infinit.net/ewan/screen_shot-50439.gif (error at bottom)

I looked at the reviews for the Kindle edition of My Horizontal Life and nobody mentions it. Am I the only one to experience this?

I'm hoping the Kindle update will fix it (and they would need to update their PC software as well) if the problem is software related. But if it's data related...  You'd think their conversion software would be sophisticated enough to handle special characters in books that are not complex in terms of character set. Come to think of it, I seem to recall seeing an "é" in one of the books I've read on my Kindle... *sigh*


Claude

[marianneg]

I've installed Kindle for PC and opened My Horizontal Life. The same odd characters are displayed in the same spots, same missing hyphen in Judas Unchained (thanks for not including a text search function in the PC version!!!) so I'm guessing it's not a Kindle (the device) problem. The files I have are like that and contain those errors.

Well, I'd be surprised if Kindle for PC supported a larger character set than the Kindle device.  What would be the point?  But, yeah, I think it's a case of bad conversion and/or sloppy (or non-existent) proofreading.

[ClaudeM]

Why not use the computer's character set?

I just finished the 2.5.2 update. It didn't fix the problem. 

If the problem is inside the files themselves, Amazon or the publishers would need to fix them.

[nogdog~6op6ou]

From http://forums.digitaltextplatform.com/dtpforums/entry.jspa?externalID=281&categoryID=20#4-3_What_characters_are_supported_in:


4-3 What characters are supported in DTP?

Amazon Digital Text Platform supports text in the 'Latin-1' format and all characters in that character set. The only characters from that set not currently supported are: spades, clubs, hearts, up-arrow, down-arrow, alpha, beta, gamma. That character set includes the most characters found in western European languages and many associated glyphs such as the 'copyright' mark, superscript (power of two) and German umlauts. You can see a list of all supported characters here. An image showing all those characters can also be seen here.

Please note that the Amazon DTP interface will not automatically recognize 'illegal' characters in your input, such as Eastern European characters, Japanese letters, Russian characters, etc. You will therefore be able to enter unsupported text, especially in the fields for authors, content title, and other product information.

When previewing your content after uploading it, illegal characters will not be displayed correctly in the content body. They will, however, be ignored in the product information (see above). This means that it is possible for you to accidentally enter unsupported characters in the product info and make your content available for purchase, displaying incorrect or garbled characters on the Amazon Kindle Store product page. Please be sure to check that all the text you enter in the product info is therefore compliant with the Latin-1 character set.


Not saying this is necessarily the cause in each case, but if the author/publisher submitted a file with non-Latin-1 characters in it and then did not proof/correct the result, you could end up with "junk" characters.

[ClaudeM]

Thanks for the info! I'm glad to know the Kindle supports just about every character a regular book would use.

How unlikely is it that American authors would include non-Latin-1 characters in their books, particularly when it's clear the odd characters are replacing something fairly simple?

In the "Why not}" example, the } is most likely replacing a question mark.

[nogdog~6op6ou]

Thanks for the info! I'm glad to know the Kindle supports just about every character a regular book would use.

How unlikely is it that American authors would include non-Latin-1 characters in their books, particularly when it's clear the odd characters are replacing something fairly simple?

In the "Why not}" example, the } is most likely replacing a question mark.

Yeah, that was one of the things that raised the "optical scanner error" flag for me originally, as I could sort of see how a "?" in some font, perhaps slightly damaged on the scanned page, might get interpreted as a "}". But it still feels like a stretch. About all I can say with some degree of assurance is that whoever generated the Kindle version of that book did not do a very thorough job of proofreading and correcting it.

[Geoffrey]

I looked at your first example (http://pages.infinit.net/ewan/screen_shot-50435.gif ) in my copy of the book and it looks fine  I have a proper italicized question mark ..... I have fewer locations though - so I looked at the book in the Manage your kindle and I have a different, older version than the one for sale now. 

I wonder what they fixed?   

[ClaudeM]

Thank you, NogDog and Geoffrey.

Geoffrey, I set the characters fairly big because I tend to read in bed without my glasses.

Could you run a search for "pregnant" to see if you can find "while you're pregnant ... ?/ you're"?


Claude

[scott343]

This is happening to many text files I have copied to my Kindle, also. The files are missing letters or have strange symbols.

I compared the versions on my computer with the ones on the Kindle using a file compare program, and the files are exactly the same.

However, on the Kindle screen, it has missing letters or strange symbols.

My only assumption is that the Kindle can't handle TXT files which have accented letters (such as maître d'). It messes up the indexing somehow - not where the accented letter is, but somewhere else.

I'm looking into other alternatives, if anyone is interested.

[ClaudeM]

I've read quite a few Kindle books since "My Horizontal Life". That one was the worst in terms of odd characters.

One character that is regularly missing from the Kindle editions is the hyphen. I've noticed that in several books.

A few weeks ago, I sent feedback to Amazon about a typo in the title of a book I was reading. The error was in Amazon's database, not in the book itself, but it was annoying to see the error in my list of books (Amazon fixed it. Yayyy!). While I was at it, I mentioned that the book (several I've come to notice, actually) contained a table of contents but the "Go to" menu option did not offer the option of going to it. I was told that the problem was on the publisher's end.

I think the program they use to convert the original manuscript to the Kindle format still has a few bugs...