Massively Parallel Procrastination

Things I ought to be blogging about

I just got back from OSCON.  I gave an Ignite talk about my Kindle hacking, a "regular" 45 minute talk about SD, the p2p bug tracker that I work on and a half-day talk on RT.  OSCON, as always, was amazing. I think I only made it to a half-dozen actual sessions - the hallway track kept me that busy.

About a dozen people yelled at me for not doing more to publicize some of the projects I'm involved with. Each and every one of these deserves a whole series of blog posts. Maybe by listing them here, I'll embarass myself into writing a bit more.  

I've had a busy year.  I didn't realize how busy until I wrote this post.  Below are some of the projects I've been involved with / instigated. In almost every case, I've had fantastic collaborators who have helped make the tools a reality. But the projects are all my fault ;)

SD is a peer to peer bug tracker (think git for bugs) that can also sync to RT, Hiveminder, Google Code, GitHub, Trac. Read-only support for syncing with RedMine is available today, with full two-way sync coming soon. Sync plugins are only a few hundred lines of code once you have a CPAN module to talk to an app.  For now, you can check out the talk I gave at OSCON.

K-9 is an opensource email client for Android.  I founded the project when I forked the core android "Email" app to get some needed bugfixes onto user devices quickly. Since then, K-9 has added features and fixes at a pretty rapid pace. Best of all, other folks do a lot of the work :)

Shipwright is a build and distribution system for applications. It has special features designed to tame Perl's CPAN, but it works well for non-Perl apps too.  It lets you build a versioned source archive of your application and all its dependencies. It comes with tools to help you distribute buildable source archives with an app and all its dependencies (down to libc if you swing that way) with a single installation tool.  The resulting binary packages have a bit of magic to make them magically relocatable. Just recently, we also released support for multi-architecture binary distributions.

Changelogger is a tool we've been working on at Best Practical to ease the ordinarily tedious process of creating a human-readable changelog for your software package out of the version control system's commit log.  It lets you open the change categorization and tidying process to a community of developers who can vote on rephrasings and categorization of changelog entries. The tool let Nicholas Clark slice through the 2500 changes included in Perl 5.10.1 in about 7 hours.  Once we get a few more issues cleaned up, we'll be running a changelogger as a public service.

Kindle hacking.  At this point, I have an ubuntu Jaunty Jackalope chroot with working X server, Keyboard and 5-pad.  I need to fix an X server bug, then I'll build a downloadable installer and let the rest of the world start playing around.

I've done a bit of minor javascript hacking to make Twitter's web view more palatable to me: http://fsck.com/~jesse/spyhunter/ and http://fsck.com/~jesse/narrow-twitter.user.js.

At YAPC::US last month, I came up with a pair of 20 line scripts which let me read twitter in Mutt. I get keyboard bindings, proper threading, lightning fast searching AND I can keep it running in screen ;) It's actually just a replacement for /usr/bin/sendmail and some goo to turn tweets into maildir messages.  My plan with this is to clean it up and add RSS feeds, Facebook and a few other activity streams. I may then try to package it up or set it up as a service. The code's all in my github account.

A half-dozen other things I tried out died on the vine: A postgres-based email archival tool, A ground-up rewrite of RT (don't worry, RT4 is alive and well, but that's another post), cleanup of MobiPerl and...well, I've blocked out my memories of the rest of them.

A new definition of wrong

I haven't been blogging about the Kindle for a while...mostly because I haven't hacked on the Kindle in a while.  The release of the DX got me excited about what's possible with a device like the Kindle. I spent a bunch of time trying to get native framebuffer applications working on my Kindle 2. Yesterday, I struck upon an awful, awful idea.

Ubuntu Jaunty has an ARM target.  It's a fairly similar linux to the OS shipped on the Kindle 2 and the Kindle DX. A little bit of fiddling and an NFS export later, I was able to chroot into an Ubuntu environment on my Kindle. 

That was when I discovered that Lab126 have built the Kindle's kernel without CONFIG_VT...and X.org really wants a tty or virtual terminal, but not for any particularly good reason.

This evening, I managed to bludgeon X.org into submission and found myself face to face with everybody's favorite checkerboard background.

After that, it was just a hop, skip and an apt-get until I was watching the mesmerizing transitions of xdaliclock.

(Yes, I know it's incredibly blurry. I haven't managed to get the pinhole camera on my DX calibrated yet)

Clearly this is the killer app for the Kindle just a stepping stone. I still don't have the Keyboard or 5way hooked up and what I do have working is incredibly brittle.  But xpdf (and lots of other stuff) runs unmodified.

Improved PDF rendering for Savory

Now with better default fonts (Google's free Droid fonts) and border cropping.

http://savory.googlecode.com/files/savory-image-2009-04-10.zip

Savory Screenshots

Over the past day, a few readers have asked me what converted PDFs look like on the Kindle 2 and how readable they are. (One of you even said you'll buy a Kindle 2 if I post some screenshots.) And yes, Amazon pays me a referral fee for every Kindle 2 I sell. So far, I've lured two of you in ;)

All the screenshots below have been reduced. Click on them to see full-size versions.

The Savory conversion process

 

 

A scientific paper, converted from PDF

Fit to screen:


Fit to height (yeah, height. It's rotated):

Fit to screen

Full size:

Fit to height:

Playing to the crowd

From the bits of feedback I've gotten on Savory's initial release on Friday, it seems pretty clear that people want to be able to view rendered PDFs that look "like they're supposed to" 

Savory's initial release used a PDF-to-HTML converter that extracts the text from a PDF and turns it into a reflowable ebook.  If your PDFs are generated from text by modern tools. this is great.  Most PDFs aren't.

Over the weekend, I put together a PDF renderer for Savory build on Poppler and the Kindle's built in Manga support (If you have a .zip or .cbz file full of images, the Kindle will display the images like a book)

For now, every PDF is converted twice. Once to text and once to a set of rendered images. So far, it's working pretty well for me. I'd love some feedback on savory-image-beta-2009-04-06.zip from anyone who's game.  Just replace your regular savory image with it and reboot the kindle.

What's next for Savory

• Good conversions of image-heavy or image-based PDFs. (I just need to build Ghostscript on the )
• A revised installer/uninstaller based on some suggestions from the folks at Blog Kindle (Who have a Unicode font mod for the K2 up at http://www.blogkindle.com)
• Tracking down slightly a few unreliable ebook conversions/installations I've noticed.

Savory

UPDATED 5 April 2009 - 11:16 EDT: Pointers changed to Savory installer and uninstaller 0.06 which provide minor run-time reliability and book-detection fixes. If you are running 0.05, you should uninstall the old version and install 0.06.

UPDATED 6 April 2009 - 9:41 EDT: I put together beta-quality support for picture-perfect PDF production. You can try it out if you promise to report back

UPDATED 6 April 2009 - 22:44 EDT:Screenshots of savory's improved PDF converter.

---

Over the past few weeks, I've spent much of my spare time with my new Kindle2...When I bought it, I was excited to have a gorgeous, solidly built ebook reader backed by Amazon's catalog. I figured I'd end up reading a bit more than I already did and spending a bit more money on books than I already did. Both turned out to be true. What I didn't count on was finding a new hobby.

Hi, I'm Jesse and I have a software problem.

When I get a new bit of hardware, I often end up writing software for it. In 2001, when I got one of Canon's first Digital SLRs, I found myself writing code to extract usable images from the "RAW" image files the camera produced. This fall, I picked up a new T-Mobile G1 to replace an iPhone...and found the email client lacking. The Android platform that the G1 runs on is free and open. So I dusted off some long-unused Java experience and created K9, an enhanced version of Android's email client.

I really didn't think this would happen with the Kindle. I was wrong.

What is Savory?

Savory is a native ebook conversion package for the Kindle 2. It lets you download and read PDFs and ePubs on the Kindle without a manual conversion step.

So, you hacked the Kindle?

No. There have actually been a number of other user-generated Kindle updates. igorsk created the toolset which generates Kindle and Kindle2 updates. Other members of the Kindle community have created Kindle2 updates which change the Kindle's fonts to support books in non-western language, let you set your own screensavers and a bunch more. These packages already contain everything a technically savvy user would need to install software on the Kindle. What I did was to port an ebook-conversion package to run reasonably efficiently on a 500mhz ARM with 128 megabytes of system memory and to write a small program which watches for new ebooks in a few chosen formats and run those through the conversion tool.

Does Savory let me decrypt books Amazon has protected with DRM?

No

So this is like KindlePid?

Nope. Not at all. KindlePid is a tool for reverse-engineering your Kindle's "Mobipocket Pid." KindlePid lets you buy DRM-protected ebooks from providers other than Amazon. Savory converts PDFs and ePubs that you download to your Kindle over WhisperNet or 3G into unprotected .mobi documents. There are web-based and desktop tools which can do everything Savory can do. Savory just brings these features directly to your Kindle 2.

Does Savory let me read DRMed ebooks?

No. Savory does not include support for ebooks protected by DRM. DRM is an incredibly "hot" topic in the ebook world right now. There are varying opinions on its efficacy. My opinions on the matter aren't relevant, except to say that I am not touching the topic with a 10 foot pole. It will not convert DRM-protected ebooks into a format the Kindle will read. It will not add or remove DRM from any ebook.

Does Savory let me do anything Amazon didn't already let me do for free?

No. It just makes some things that are a little cumbersome out of the box simple and transparent. It's already possible to use desktop and web based services to transcode ePub and PDF documents into the Kindle-compatible Mobipocket format. Amazon also provides both free and for-pay email-based conversion services you can use.

Does Savory work with the Kindle 1?

No.

Why did you create Savory?

I'm in love with my Kindle. I've been reading ebooks on screens of various sorts for many years, but the Kindle2 is the first device that I actually enjoy reading as much as I enjoy reading paper books. I've tried other ebook readers, but for a variety of reasons, they just don't work for me. My goal is to make it easier for readers to read more free content on the Kindle.

I got the idea after reading Tim O'Reilly's editorial in Forbes about why the Kindle platform will be more successful if it's more open. My first experiments were actually in server-based transcoders to convert PDFs and epubs to Kindle-compatible Mobipocket books, but I quickly realized that running the converter locally on the Kindle would result in a much better user experience and make the Kindle more useful. And yeah, it seemed like an interesting project.

What document formats does Savory let me read?

Savory allows you to read .epub and .pdf files on your Kindle. It does this by converting these documents to Mobipocket format ebooks using Calibre. You should note that the version of Calibre Savory uses only works with text-based PDFs. If you have image-based or scanned PDFs, conversion will fail. (Images in your PDFS are ok. There just needs to be some text to extract.) Updated: Screenshots of the new and improved PDF converter.

How does Savory work?

Plain-english version

Savory installs a small program which runs on your Kindle and watches for new files in the 'Documents' directory with names ending in '.epub' and '.pdf'. When the system notifies Savory that a document has shown up, it wakes up and runs an open-source file conversion program called Calibre. Savory also updates your Kindle2's browser configuration file to tell it that the Kindle can now handle .pdf and .epub documents.

Technical version

Savory changes the browser configuration file to allow download of pdf and epub documents. It adds a new "init" script which tries to mount savory-image.ext3 on boot. If that succeeds, it runs bin/savory_daemon from within the image mentioned above. savory_daemon is a Python script that watches the documents/ directory and invokes a converter based on Calibre when it sees something that looks right.

Is Savory supported by Amazon?

No. If you have a problem with Savory, DO NOT CONTACT AMAZON. This isn't software they wrote. They're not responsible for it. Please don't bother them about it.

If the nice folks at Amazon contact me (you can find my email address at the bottom of this FAQ) and ask me to stop distributing Savory, I will do so. My goal isn't to "hack" the Kindle, deprive Amazon of revenue or place an increased support burden on Amazon's Kindle team. I just want to make the Kindle2 an even more useful reader than it already is.

Does Savory void my warranty?

I don't know. If you're not comfortable with the possibility, do not install Savory.

Does Savory come with any guarantee or warranty?

I guarantee that Savory will not give you the ability to fly or to see through walls. Past that, no. Savory comes with no warranty or guarantee OF ANY KIND. If it causes your Kindle to burst into flames or gives your pets the ability to read, you are ENTIRELY ON YOUR OWN. By downloading and installing Savory, you accept full responsibility for anything it does.

Savory is distributed under the MIT license:

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

What do I do if Savory breaks my Kindle?

Head on over to the MobileRead Kindle forums and post your sob story. Hopefully, someone there can help figure out what went wrong and help you get back to a working state. Remember that Savory comes with absolutely no warranty and everybody on MobileRead who might help you out is a volunteer.

How do I install Savory?

Installing Savory or any other third-party update on your Kindle may destroy your Kindle. If that happens, you will have a $360 paper weight. DO NOT INSTALL SAVORY UNLESS YOU'RE WILLING TO END UP WITH A DEAD KINDLE.

You'll need to download three files:

savory-image.zip

This file contains Savory's file-conversion system and assorted tools.

Download and unzip this file. Then, mount your Kindle2 on your desktop with USB and drag savory-image.ext3 into the "system" folder on your Kindle2.

update_RemoveSavory-0.06.bin

This file is a Kindle update which will remove an installed copy of Savory from your Kindle. Please keep a copy of this file handy in case you ever need it.

update_Savory-0.06.bin

This file is a Kindle update.

You should copy this file to the root directory of your Kindle. After you do that, unmount the kindle and click the "Menu" button, select "Settings", click the "Menu" button and click "Update Your Kindle" This will apply the update.

If "Update Your Kindle" is greyed out, mount the Kindle on your desktop again and DELETE the update file. Then, unmount the Kindle. If you're not already on the Settings page, click "Menu" and select "Settings". Make sure your Kindle is plugged into your computer with the USB cable. Click "Menu" and select "Restart". The Kindle will boot up and immediately enter USB mode. Copy the update file to the Kindle again and follow the instructions above.

I've found that the Kindle is much happier recognizing update files if you reboot the Kindle while it's connected to your computer with USB and copy the update to the Kindle before the Kindle's UI comes up at all

If you copy the file to your kindle and reboot without applying the update, it will end up in a reboot loop - I haven't figured out why yet. If that happens, hold down the "Home" button while booting to get into recovery mode. From there, you can mount the Kindle on your desktop with USB, delete the .bin file and try again.

How do I turn Savory off temporarily?

Mount your Kindle on your computer with USB. Rename the savory-image.ext3 file in the Kindle's "System" folder. Then, unmount your Kindle, click "Menu", select "Settings", click "Menu" and click "Restart your Kindle." When you later want to re-enable Savory, repeat the process, naming the file back to "savory-image.ext3"

How do I remove Savory completely?

If you've decided that Savory isn't for you or you need to remove Savory to perform an Amazon-provided system update, it's a two step process. First, you'll need to follow the instructions above about how to temporarily turn off Savory. Then, download and run the "update_RemoveSavory-0.06.bin" system update and apply it just like you applied the original system update.

Amazon's system updates very carefully check each and every file they're about to update to make sure that they are exactly as Amazon left them. This ensures that a system update doesn't unexpectedly corrupt an "important" file. Savory updates the Kindle's browser configuration to allow download of pdf and epub files, adds an "init script" to start up the savory daemon which watches for new files to convert and modifies the Kindle's "version.txt" file. This way, anyone doing support for your Kindle will know it's not running a stock system image and NOT ELIGIBLE FOR SUPPORT FROM AMAZON.

Savory copies each file it modifies before making changes. All files Savory modifies are saved with the file suffix "-beforesavory". When you uninstall Savory, it restores the pristine versions of these files.

What does Savory do to my battery life?

It makes it shorter. So far, it doesn't feel like it makes it much shorter. Comparisons and benchmarks would be appreciated

What does Savory do with the "originals" of ebooks it converts?

You'll find the originals of any book Savory converts (or fails to convert) in the savory-archive folder on your Kindle when you browse it from your computer.

Where can I get the source code to Savory?

Savory is hosted on Google Code: code.google.com/p/savory

What software does Savory use?

Savory wouldn't have been possible without kindle-update-maker, created by Igor Skochinsky.

Savory is about 200 lines of Bourne Shell and Python, built to drive a modified copy of Kovid Goyal's open-source Calibre ebook conversion and management suite.

The disk image containing the Savory daemon and the modified version of Calibre contains a number of dependencies, listed below. You can download all of these source tarballs from code.google.com/p/savory.

The patches to Calibre are available in the same source repository as the rest of Savory. All other packages are unmodified.

http://bazaar.launchpad.net/%7Ekovid/calibre/trunk Revision 2689
http://unladen-swallow.googlecode.com/svn/branches/release-2009Q1-maint Revision: 370
ftp://ftp.qtsoftware.com/qt/source/qt-embedded-linux-opensource-src-4.4.3.tar.gz
http://www.rarlab.com/rar/unrarsrc-3.7.5.tar.gz
http://www.riverbankcomputing.co.uk/static/Downloads/sip4/sip-4.7.9.tar.gz
http://sourceforge.net/project/downloading.php?group_id=68617&use_mirror=voxel&filename=rtf2xml-1.33.tar.gz&a=75791959
ftp://ftp.gnu.org/pub/gnu/readline/readline-6.0.tar.gz
http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.7.tar.gz
http://xmlsoft.org/sources/libxslt-1.1.24.tar.gz
http://xmlsoft.org/sources/libxml2-2.7.3.tar.gz
ftp://ftp.gnu.org/pub/gnu/gdbm/gdbm-1.8.3.tar.gz
http://savannah.inetbridge.net/freetype/freetype-2.3.9.tar.gz
http://effbot.org/media/downloads/elementtree-1.2.7-20070827-preview.zip
http://www.openssl.org/source/openssl-0.9.8j.tar.gz
http://git.dbzteam.org/?p=pyinotify.git;a=snapshot;h=HEAD;sf=tgz as of 2009-03-30 - pyinotify-0.8.1-py2.6.egg-info
http://sourceforge.net/project/downloading.php?group_id=45839&use_mirror=voxel&filename=pdftohtml-0.39.tar.gz&a=40219524
http://effbot.org/downloads/Imaging-1.1.6.tar.gz
ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.5.0-9.tar.gz
http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-3.1.0.tar.gz
http://wwwsearch.sourceforge.net/ClientForm/src/ClientForm-0.2.10.tar.gz
http://labix.org/download/python-dateutil/python-dateutil-1.4.1.tar.gz
http://wwwsearch.sourceforge.net/mechanize/src/mechanize-0.1.11.tar.gz
http://sourceforge.net/project/downloading.php?group_id=68617&use_mirror=voxel&filename=rtf2xml-1.33.tar.gz&a=50591281
http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c9.tar.gz#md5=3864c01d9c719c8924c455714492295e
http://pypi.python.org/packages/source/l/lxml/lxml-2.2.tar.gz#md5=b3f12344291aa0d393915e7d8358b480
http://www.imagemagick.org/download/python/PythonMagick-0.9.tar.gz
http://www.riverbankcomputing.co.uk/static/Downloads/PyQt4/PyQt-x11-gpl-4.4.4.tar.gz

How much code is Savory?

The unique parts of Savory are about 85 lines of shell script and 135 lines of Python.

Why's the Savory disk image nearly 90 megabytes uncompressed?

Savory uses Calibre, a free and open ebook conversion suite. In turn, Calibre uses Python, Qt and some other libraries. While I've slimmed down the requirements by hand, we have a long way to go before Savory fits in 5 or 10 megabytes.

What license is Savory under?

The original code in Savory is available under the terms of the MIT License. The various components bundled with Savory are all free or open software, but they're not all released under the same license.

I want to be able to use my Kindle to read books in some other format. How do I make that happen?

Savory supports a subset of the conversions provided by Calibre. If you contribute to Calibre, Savory should be able to take advantage of those improvements.

Can I run other software on my Kindle?

I don't know. Can you?

Can you help me run other software on my Kindle?

No.

The Kindle has a 3G modem in it. Can I use that to get free internet access for my laptop?

Don't even think about it. The Kindle's 3G internet access is currently provided gratis to all Kindle users for the purpose of browsing the web and downloading ebooks. Amazon could choose to start charging for this "experimental" feature at any time. On top of that, Amazon knows where you live, has your credit card number and, thanks to the Kindle's GPS, knows where you are right now.

Where can I get support for Savory?

The MobileRead forums are probably the right place to start.

So, why is it called Savory?

When I first got a my Kindle2, I experimented with a web-based transcoding proxy which lets you download ePub and PDF files to the Kindle using a tool running on a server. I called the project an "unsavory ePub hack." This new tool is a good deal more elegant and easier to use. It's no longer unsavory. That must mean it's Savory

Savory is awesome! Now I want a Kindle!

Great! That's the kind of thing I like to hear. Savory is (and will always be) 100% free. BUT! Shameless plug warning If you buy a kindle (or anything else) from Amazon, I'll earn a referral fee which goes to feed my ebook habit. Buying a Kindle 2 from Amazon through this site helps us tell Amazon that we want the Kindle to support open formats. If more than a few of you buy a Kindle through this blog, I'll post a running tally.

Who created Savory?

I'm Jesse Vincent. You can reach me at jesse - at - fsck.com. Please don't contact me directly for help with Savory, even if it sets your dog on fire or you're sure that I'll make an exception for you.

If you need help with Savory, I refer you to the forums at mobileread.com. (See above)

Yes, I am teasing you. But why am I building Qt on my Kindle?

USER    PID %CPU %MEM  VSZ  RSS TTY   STAT START  TIME COMMAND
root     1 0.0 0.2  1484  300 ?    Ss  Mar22  0:03 init [5]   
root     2 0.0 0.0   0   0 ?    S<  Mar22  0:00 [kthreadd]
root     3 0.0 0.0   0   0 ?    SN  Mar22  0:00 [ksoftirqd/0]
root     4 0.0 0.0   0   0 ?    S<  Mar22  0:00 [watchdog/0]
root     5 0.0 0.0   0   0 ?    S<  Mar22  0:24 [events/0]
root     6 0.0 0.0   0   0 ?    S<  Mar22  0:00 [khelper]
root    37 0.0 0.0   0   0 ?    S<  Mar22  0:00 [kblockd/0]
root    38 0.0 0.0   0   0 ?    S<  Mar22  0:00 [cqueue/0]
root    40 0.0 0.0   0   0 ?    S<  Mar22  0:11 [mxc_spi.1]
root    58 0.0 0.0   0   0 ?    S<  Mar22  0:00 [kmmcd]
root    76 0.0 0.0   0   0 ?    S  Mar22  0:12 [pdflush]
root    77 0.0 0.0   0   0 ?    S  Mar22  0:05 [pdflush]
root    78 0.0 0.0   0   0 ?    S<  Mar22  0:24 [kswapd0]
root    79 0.0 0.0   0   0 ?    S<  Mar22  0:00 [aio/0]
root    101 0.0 0.0   0   0 ?    S<  Mar22  0:01 [mtdblockd]
root    136 0.0 0.0   0   0 ?    S<  Mar22  0:00 [kondemand/0]
root    146 0.1 0.0   0   0 ?    S<  Mar22  3:33 [mmcqd]
root    156 0.0 0.0   0   0 ?    S<  Mar22  0:00 [kjournald]
root    190 0.0 0.1  1520  208 ?    S<s Mar22  0:06 udevd --daemon
root    384 0.0 0.0   0   0 ?    S<  Mar22  0:02 [eink_fb_bt]
root    386 0.0 0.0   0   0 ?    S<  Mar22  0:00 [eink_fb_pt]
root    390 0.0 0.0   0   0 ?    S<  Mar22  0:00 [eink_fb_sst]
root    394 0.0 0.0   0   0 ?    S<  Mar22  0:00 [pnlcd_animate]
root    583 0.0 0.0   0   0 ?    S<  Mar22  0:01 [kjournald]
root    696 0.0 0.4  1948  616 ?    Ss  Mar22  1:02 /usr/sbin/syslog-ng
root   1023 0.1 0.0   0   0 ?    S<  Mar22  1:57 [loop0]
root   1043 2.7 4.5 205616 5832 ?    Ssl Mar22 52:00 fsp /mnt/base-us /mnt/us -o rw
92    1332 0.0 0.3  2236  504 ?    Ss  Mar22  0:04 /usr/bin/dbus-daemon --system
root   1367 7.9 0.4 19080  520 ?    Ssl Mar22 147:54 /usr/bin/powerd
root   1466 0.0 0.0  1468  120 ?    Ss  Mar22  0:00 /usr/sbin/tphserver
root   1526 0.0 0.1 19012  244 ?    Ssl Mar22  0:00 /usr/sbin/volumd -d /dev/mmcblk0p4
root   1605 0.0 0.1  2652  140 ?    Ss  Mar22  0:00 /bin/sh /usr/bin/aserver.sh
root   1644 0.0 0.3  2648  396 ?    Ss  Mar22  0:00 /usr/sbin/crond -c /etc/crontab/
root   1757 0.0 0.1  2648  188 ?    S  Mar22  0:00 /bin/sh /opt/amazon/ebook/bin/start.sh
root   1793 0.0 0.1  2652  188 ?    S  Mar22  0:00 /bin/sh /opt/amazon/ebook/bin/start.sh
root   1795 0.0 0.1  2652  152 ?    S  Mar22  0:00 logger -p local2.debug
root   1821 2.8 22.2 390176 28280 ?    SLl Mar22 52:31 /usr/java/bin/cvm -Xmx16m -Xbootclasspath/a:/usr/java/lib/localedata.jar:/usr/java/lib/charsets.jar -Dsun.boot.library.path=/usr/java/lib:/usr/java/lib -cp :/opt/amazon/ebook/lib/MobiCore-impl.jar:/opt/amazon/ebook/lib/MobipocketCoreReader.jar:/opt/amazon/ebook/lib/ReaderSDK.jar:/opt/amazon/ebook/lib/SearchSDK.jar:/opt/amazon/ebook/lib/booklet.jar:/opt/amazon/ebook/lib/cd.jar:/opt/amazon/ebook/lib/framework-api.jar:/opt/amazon/ebook/lib/framework-impl.jar:/opt/amazon/ebook/lib/jdbm.jar:/opt/amazon/ebook/lib/json.jar:/opt/amazon/ebook/lib/kxml2.jar:/opt/amazon/ebook/lib/xyml.jar:/opt/amazon/ebook/booklet/AudiblePlayer.jar:/opt/amazon/ebook/booklet/AudioPlayer.jar:/opt/amazon/ebook/booklet/Browser.jar:/opt/amazon/ebook/booklet/Demo.jar:/opt/amazon/ebook/booklet/Experimental.jar:/opt/amazon/ebook/booklet/Home.jar:/opt/amazon/ebook/booklet/MobiReader.jar:/opt/amazon/ebook/booklet/PictureViewer.jar:/opt/amazon/ebook/booklet/Search.jar:/opt/amazon/ebook/booklet/TestSwitches.jar:/opt/amazon/ebook/booklet/XymlBooklet.jar:/opt/amazon/ebook/booklet/msp.jar:/usr/java/lib/libjnisystem.jar -Ddebug=1 -Dcheck_comm_stack=true -Dhttp.keepalive.timeout=60000 -Dhttp.maxConnections=16 -Dallow_demo=false -Dawt_fb_enable=0 -Dextkeyboard=false -Dconfig=/opt/amazon/ebook/config/framework.mario.conf -DPLATFORM_CLASS_FILE=/opt/amazon/ebook/config/platform.conf -DENABLE_SEARCH_INDEXING_THREAD=true com.amazon.ebook.framework.Main
root   1892 0.0 0.1  2100  188 ?    Ss  Mar22  0:19 /usr/local/bin/telnetd -p 2323 -l /bin/sh
root   1929 0.0 0.1  1056  176 ?    Ss  Mar22  0:00 /opt/local/unsavory/sbin/dropbear -d /opt/local/unsavory/etc/dropbear_dss_host_key -p 22
root   1959 0.0 0.1  2652  148 ttymxc/0 Ss+ Mar22  0:00 /sbin/getty -L 115200 ttymxc0 -l /bin/login
root   2365 0.1 0.0   0   0 ?    S<  Mar23  0:29 [rpciod/0]
root   5155 0.0 0.4  2652  584 pts/5  Ss  00:54  0:00 /bin/sh
root   6826 0.2 0.6  2824  880 pts/5  S+  02:40  0:05 /bin/sh ./configure --prefix=/opt/savory -qt-freetype -no-cups -no-nis -qt-libjpeg -qt-libpng -qt-libtiff -no-opengl -depths 1,4 -no-qt3support -no-webkit
root   7707 0.0 0.0   0   0 ?    S<  Mar23  0:05 [loop5]
root   7708 0.0 0.0   0   0 ?    S<  Mar23  0:00 [kjournald]
root   8503 0.0 0.0   0   0 ?    S<  Mar23  0:00 [loop3]
root   8617 0.0 0.0   0   0 ?    S<  Mar23  0:00 [loop4]
root   11603 0.1 1.0 28336 1392 ?    Ssl 02:57  0:02 /usr/bin/monit -Ic /etc/monitrc
root   11856 0.0 0.3  1460  408 ?    Ss  02:58  0:00 /usr/sbin/watchdogd
root   12366 0.0 0.9 84160 1188 ?    Sl  Mar22  0:01 audioServer -I
root   12484 0.1 0.7  3656  956 ?    Ss  02:59  0:01 /opt/local/unsavory/sbin/dropbear -d /opt/local/unsavory/etc/dropbear_dss_host_key -p 22
root   12485 0.0 0.5  2652  720 pts/0  Ss  02:59  0:00 -sh
root   12489 0.0 0.7  3668  924 pts/0  S+  02:59  0:00 /mnt/us/unsavory/bin/screen-4.0.3 -x
root   12490 0.0 0.5  2652  712 pts/1  Ss  02:59  0:00 /bin/sh
root   15956 0.0 0.0   0   0 ?    S<  Mar22  0:09 [charger/0]
root   23310 0.0 0.5  2652  712 pts/2  Ss+ 03:09  0:00 /bin/sh
root   23469 0.0 0.5  2652  720 pts/3  Ss  03:09  0:00 /bin/sh
root   23598 0.0 0.0   0   0 ?    S<  Mar23  0:00 [loop1]
root   24135 0.0 0.0   0   0 ?    S<  Mar23  0:00 [loop2]
root   25652 0.0 0.5  2652  716 pts/4  Ss  03:14  0:00 /bin/sh
root   26479 0.2 0.7  2532  960 pts/4  S+  03:15  0:00 make
root   26482 0.0 0.4  2648  584 pts/4  S+  03:15  0:00 /bin/sh -c cd src/tools/bootstrap/ && make -f Makefile 
root   26483 0.1 0.8  2572 1068 pts/4  S+  03:15  0:00 make -f Makefile
root   28838 0.0 0.4  1888  628 pts/4  S+  03:21  0:00 g++ -c -pipe -g -fno-exceptions -O2 -fPIC -Wall -W -DQT_BOOTSTRAPPED -DQT_LITE_UNICODE -DQT_NO_CAST_FROM_ASCII -DQT_NO_CAST_TO_ASCII -DQT_NO_CODECS -DQT_NO_DATASTREAM -DQT_NO_GEOM_VARIANT -DQT_NO_LIBRARY -DQT_NO_QOBJECT -DQT_NO_STL -DQT_NO_SYSTEMLOCALE -DQT_NO_TEXTSTREAM -DQT_NO_THREAD -DQT_NO_UNICODETABLES -DQT_NO_USING_NAMESPACE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -I../../../mkspecs/qws/linux-generic-g++ -I. -I../../../include -I../../../include/QtCore -I../../../include/QtXml -I../../3rdparty/zlib -I.uic/release-shared-emb-generic -o .obj/release-static-emb-generic/qiodevice.o ../../corelib/io/qiodevice.cpp
root   28839 49.5 24.5 36116 31160 pts/4  R+  03:21  0:09 /mnt/us/savory-toolchain-4.1.2/bin/../libexec/gcc/arm-unknown-linux-gnueabi/4.1.2/cc1plus -quiet -I../../../mkspecs/qws/linux-generic-g++ -I. -I../../../include -I../../../include/QtCore -I../../../include/QtXml -I../../3rdparty/zlib -I.uic/release-shared-emb-generic -iprefix /mnt/us/savory-toolchain-4.1.2/bin/../lib/gcc/arm-unknown-linux-gnueabi/4.1.2/ -D_GNU_SOURCE -DQT_BOOTSTRAPPED -DQT_LITE_UNICODE -DQT_NO_CAST_FROM_ASCII -DQT_NO_CAST_TO_ASCII -DQT_NO_CODECS -DQT_NO_DATASTREAM -DQT_NO_GEOM_VARIANT -DQT_NO_LIBRARY -DQT_NO_QOBJECT -DQT_NO_STL -DQT_NO_SYSTEMLOCALE -DQT_NO_TEXTSTREAM -DQT_NO_THREAD -DQT_NO_UNICODETABLES -DQT_NO_USING_NAMESPACE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE ../../corelib/io/qiodevice.cpp -quiet -dumpbase qiodevice.cpp -mcpu=arm1136jf-s -auxbase-strip .obj/release-static-emb-generic/qiodevice.o -g -O2 -Wall -W -fno-exceptions -fPIC -o -
root   28840 0.5 3.5  5828 4536 pts/4  S+  03:21  0:00 /mnt/us/savory-toolchain-4.1.2/bin/../lib/gcc/arm-unknown-linux-gnueabi/4.1.2/../../../../arm-unknown-linux-gnueabi/bin/as -mcpu=arm1136jf-s -meabi=4 -o .obj/release-static-emb-generic/qiodevice.o
root   28994 0.0 1.7  7404 2248 pts/5  R+  03:21  0:00 /var/tmp/maemo-build/src/qt-embedded-linux-opensource-src-4.5.0/bin/qmake -spec /var/tmp/maemo-build/src/qt-embedded-linux-opensource-src-4.5.0/mkspecs/qws/linux-generic-g++ -o /var/tmp/maemo-build/src/qt-embedded-linux-opensource-src-4.5.0/./examples/network/fortuneserver /var/tmp/maemo-build/src/qt-embedded-linux-opensource-src-4.5.0/examples/network/fortuneserver/fortuneserver.pro
root   28995 0.0 0.7  2644  992 pts/1  R+  03:21  0:00 ps auxwww
root   32292 0.2 1.0  3896 1272 ?    Ss  Mar23  1:15 /mnt/us/unsavory/bin/SCREEN-4.0.3

More fascinating Kindle 2 tidbits

There appear to be on-board PPP configurations for both Sprint and AT&T (though of course the Kindle's EVDO modem will only talk to Sprint).

As indicated by a previous anonymous commenter, it's not actually that hard to turn on the 3G modem while USB networking is on.

Sorry, kids. It's my sad duty to report that Amazon and Lab126 are neither inept nor stupid. You can't get much of anywhere except for Amazon or their web proxies. Your dreams of a magic, free internet dongle the size of a paperback book that works anywhere in the US will have to wait.

One slightly terrifying thing I noted in the NetFront configuration file on the Kindle 2 - All traffic is proxied through fints-g7g.amazon.com, Amazon's Kindle web proxy. HTTP and HTTPS alike are proxied on port 80. Amazon can see what you're downloading, even if you "use SSL." (As could anybody who could sniff your EVDO traffic, but I'm told that's something that's only easy if the attacker is running MovieOS).  I know there are solid technical reasons for this decision on Amazon's part. It doesn't exactly make me comfortable.

What else did I discover during my week away? This is a rough first pass at a list.

There's code infrastructure for iphone-esque orientation switching, though I'm not seeing anything that suggests that the backend for that exists...yet.

Building an entire GCC toolchain that can run natively on the Kindle (don't ask why just yet) is a real pain in the ass, but thanks to the Fedora ARM nfs-root instructions, the Ubuntu ARM root filesystem and crosstool-ng, I think I may have something stable and reproducible soon.

I'd previously cross-compiled gcc, binutils, glibc and coreutils by-hand, but it's turned out not so stable.  Getting somewhat desperate before I managed to find the right incantation to overcome Q-emu/ARM's weird SCSI lockup problems, I actually got enough of a dev environment up on my N810 to build an ARM-native Perl linked against the ~right glibc such that it runs ok on the Kindle.

Gone Fishin'

As a heads up, I'm going to be in the UK for the next week for work, so you shouldn't expect to hear much out of me on the Kindle front.

I have a plan, but don't really want to say anything until I can prove to myself that the plan will work. Maybe if you corner me in an epub over a pint.