Skip to content

Self-publishing a book, Part 2: My tool chain

December 24, 2011
Part 1: Why and How

As a Python programmer, I’m of course biased towards tools that are written in Python or by Pythonistas or at least feel “Pythonic”. I also had a set of requirements:

  • The tool chain had to somehow support testing all or almost all the code.
  • It had to be able to produce some sort of print-ready format, which in practice means PDF.
  • Being able to make HTML would be an added bonus, as I wanted the option to make a website out of the book.

The obvious choice to write this book was therefore to use ReST, and Sphinx to generate PDF/HTML from that. Sphinx goes via LaTeX to generate the PDF and writing it directly in LaTeX would have been an option, it’s used for a lot of technical documentation. But I didn’t feel like learning LaTex, and I didn’t know how good the LaTeX to HTML tools were.

First problem: Testing the code

The “canonical” way to test ReST code is to write it as doctests and then run the ReST source code with a testrunner. As Brandon would accept ReST for Python Magazine, this is how I wrote the articles. This had various problems, most of which Manuel promised to solve, but Manuels syntax for what makes up a doctest is different from Sphinx syntax. I solved this by writing a plugin for Manuel to handle the Sphinx syntax so you could use Manuel to run tests in Sphinx.compatible ReST, but it was relegated by Benji to a branch, where I think it died.

Another problem was that I needed to mix Python 2 and Python 3 code. I had originally intended to have a flag for what versions of Python it should be run with, and let Manuel skip tests is the test wasn’t in the list of Python versions. But I didn’t want to have my own branch of Manuel that only I used, so in the end I decided to drop Manuel and go for a much simpler solution. This solution wouldn’t have worked well for the articles, but it works fine for the book, and that is to use the literalinclude directive. This simply includes an external file, as a literal block.

This way I wrote (nearly) all code examples in the book as separate files, usually doctests, and included them into the book with literalinclude. I then wrote one unit test for each Python version, where I made a DocFileSuite out of all the files that should be tested for that version. The test for Python 3.1 will also run 2to3 on some of these code snippets first. I then have a simple bash file that will run all the unit tests.

Second problem: LaTeX

As mentioned above, Sphinx looked like a promising solution to not have to learn LaTex or other complex publishing softwares. That turned out to be wrong, I still had to learn a lot of LaTeX. The first reason for this is that I wanted to be able to select what fonts I used. So one of the first questions was how to select other fonts in LaTeX. And the answer is: You don’t really. It turns out that LaTeX is just a set of TeX macros to make it easier to use TeX, which is highly involved. LaTeX in turn comes in several different distributions, which in turn include different sets of macros, called “packages” and different PDF generators. And each distribution typically includes several different packages that claim to do much the same thing, such as selecting fonts.

Hence, when you Google for how to do this, you get several different answers, one which is actually a good working answer, and some that may be technically correct but may lead you completely astray and only give you more problems.

For Linux, the distribution to use is TeX Live. It’s a sort of mega-distribution that merged all other distributions, if I understand correctly. Although it’s the one included with almost all Linux-distros that may not help you very much, I ended up having to download TeX Live and compile it from source (which takes ages) because of bugs rendering the version included with Ubuntu at that time unusable for me.

And then I had to learn enough LaTeX to change the default templates with Sphinx to do what I wanted. I needed to not only include a “preamble” of LaTeX code in the Sphinx configuration, but also modifying sphinx.sty and sphinxcustom.cls. This was mostly changes in design and margins etc. I did have to fix some bugs in Sphinx usage of LaTeX as well. I still have to feed those bugfixes back to Sphinx, I’m sorry I haven’t done that. Soon, I promise.

Third problem: Fonts

Even after I had figured out that the correct way to select fonts is \usepackage{fontspec} I still had problems. One of my technical reviewers pointed out that the PDF didn’t have any ligatures. That was indeed strange, and after much fiddling around I figured out that the version of the font I was using simply did not have any ligatures. In the end I ended up using TeX Gyre Schola, an open source version of Century Schoolbook for the main text, the also free DejaVu Sans Mono for the code, and the completely commercial Flux Bold for headings. I went with Flux Bold for the headings because it is beautiful, used in the Python logo, and I already owned a copy anyway. DejaVu Sans Mono was selected purely on aesthetic reasons and just happened to be free, and TeX Gyre Schola was selected as a version of Century Schoolbook that worked, with ligatures. That it was free was again pure luck.

The tool chain

So the complete tool chain looks like this:

  1. ReStructuredText
  2. Python tests using unittest and doctest
  3. Sphinx to generate LaTeX output
  4. xelatex to generate PDF

Out of this comes a PDF file that makes up the inside of the book. Another PDF makes up the cover, and you upload both to CreateSpace, and they will make a book out of it!

I used Sphinx to generate the book site as well. I could probably have done it faster in Plone, but I decided to use Sphinx to generate a static HTML site because that made it easy for me to put up a sample chapter, and also meant I held the door open to easily put all of the book on the web at a later date (which I indeed also ended up doing, more on that in a future installment).

Self-publishing a book, Part 3: Editing and Reviewing

About these ads

From → book, python, python3

  1. Lennart! You are awesome! Thanks for this!

    I’m on the edge of my seat for the next installment.

    Merry Christmas!


  2. Merry Yuletidings to you too! Each of these took months to finally write, so don’t hold your breath. :-)

  3. Zhang permalink

    Hi! I have been exploring how to generate pdf from sphinx for days, but failed. Many people mention the rst2pdf solution, but it seems this is not a mature tool (actually i never succeeded in pdf generation using rst2pdf). Then i found that some people mention that the texlive toolchain works. However, the problem is, i’m a Windows guy, and it seems most of the related info i’ve found on Google doesn’t help. I’ve install texlive (the full installation pacakge) on my windows xp, but have no clue on how to modify the for integration (and i just fonnd that there are other files needed to be modified, as discribed in your post). Would you be kind enough to write more details on this aspect?

  4. I don’t use Windows, so I can’t help you, sorry. Make sure all the text tools are in your path, though. After that the default Make file should work.

  5. Zhang permalink

    Sorry to hear that. Anyway, thanks for your reply and these nice posts, i guess i should figure it out in other ways.

  6. Ok, now where is that part 3 you promised? :-) You suggested earlier that you’d talk about the monetary stuff (“polish banking system”), so I’m hoping for some economic blog post. How many did you sell? Any indication? How did the creative commons stuff work out? Any tips for other authors?

Trackbacks & Pingbacks

  1. Self-publishing a book, Part 1: Why and How « Lennart Regebro: Python, Plone, Web
  2. Self-publishing a book, Part 3: Editing and Reviewing « Lennart Regebro: Python, Plone, Web

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 1,225 other followers

%d bloggers like this: