And give the talk I did (slides). Because the students had the day filled with lectures, to avoid boring them I gave this one in a more provocative style than I might otherwise have done (and hence the name of this post—and of the talk’s unofficial title). The discussion was indeed very lively, and in fact, at times it became more heated than what I had expected. In retrospect though, I did one mistake. When arguing these things, it is usually necessary to systematically deconstruct a number of unexamined assumptions the audience naturally brings to the discussion. But because I accepted (and indeed posed) questions during the talk, that deconstruction got interrupted again again, severely derailing the course of the talk I had planned. In retrospective, I should not have allowed questions, except at the end.
Nevertheless, I am grateful for all the feedback I have received. One of most poignant remarks, was actually made by a colleague of mine, who also attended the lecture. He pointed out that I do not have all the details of how things should function if we ditch copyright in particular, or intellectual property in general (which is correct); and thus I should not outright dismiss them, at least not before being able to provide a suitable alternative. This last part however, is not correct, and here is why.
As far as I can tell, all individual rights are (or can be thought of as) restrictions on the behaviour of everyone else (other than the individual in question). Thus my right to life is a restriction on the behaviours of everyone else, prohibiting them from depriving (or attempting to deprive) me of my life. The same is true of freedom of speech—it is a restriction that prohibits behaviours from everybody else (historically this meant governments), aimed at silencing me whenever they don’t like what I am saying—freedom of religion, and so on.
Put another way, one individual’s rights are always abridgements on the liberty of others. But it falls on the proponents of a given right, to show that the corresponding abridgement of liberty is necessary. It is not those whose liberty is to be putatively abridged that need to show why it ought not to; or that have to provide “suitable alternatives”. Historically this has always been so; in fact J.S. Mill’s influential essay, On Liberty, which I quoted in the lecture, was written precisely to justify why the rights he supports should indeed be implemented. Just to mention one example, he defends the right to freedom of expression, not because the State—that in 19th Britain was the major threat to that freedom—had failed to show that it had any reason to suppress it, but rather because said freedom is vital to the full development of the individual, and that this is sufficient to justify limiting (restricting) the State’s actions in this particular regard.
Back to copyright and ilk, if they are to be construed as author’s rights, then it falls on the proponents of said rights to show why the rest of us should tolerate the corresponding abridgements of liberty—and most definitely not the other way round! Insofar as I know—and yesterday’s lecture seems to have confirmed it—that reason boils down to economics: how should authors make money, without the aid of copyright et al.? I attempted to show, on both the lecture and the essay, there’s mounting evidence that that would not be a problem at all. And thus there is no need whatsoever to abridge anything. But since we are talking about rights, I might as well address the deeper flaw afflicting that copyright-because-economics rationale.
“Author rights” such as copyright are, unlike their moral rights, established for the sole purpose of enabling (or easing) business transactions. But their effective enforcement can only be done by trampling over far more fundamental rights, two of which are the right to privacy and the right to due process (if the reader disagrees, please feel free to provide an effective counterexample enforcement mechanism; I know of none). The usual retort to this is to say that we need to find a “balance”, between the interests of authors and friends, and the rest of society that would like to maintain said fundamental rights intact. This is however, woefully misguided—and I can show so by recalling a similar, albeit far more extreme, example of the same reasoning, that took place in the Southern states of the U.S., in the early-to-mid 19th century.
Back then, the economy of those states was largely based on slave labour. When the Abolitionists began propagandising the, well, abolition of slavery, what did the defenders of the status quo (meaning slavery) replied? That «the sudden end to the slave economy would have had a profound and killing economic impact in the South where reliance on slave labor was the foundation of their economy. The cotton economy would collapse. The tobacco crop would dry in the fields. Rice would cease being profitable»^{3}. This seems outrageous to us today, because we hold the right to freedom to be so fundamental, that it vastly outweighs any economic considerations. In other words, freedom comes first, and if that damns business, then so be it. But if, for some reason, one were to become convinced that freedom actually isn’t that fundamental, then the argument espoused by the Southerners suddenly becomes a lot more palatable.
And that is precisely the problem with the thesis of “finding balance” between authors and society at large. It seems palatable, only because most of us still do not realise how much the enforcement of those “IP rights” imperils rights that are far, far more fundamental. Maybe this is because the net and computers are still relatively new mediums; but be that as it may, those rights should not be sacrificed for the mere sake of economic convenience. And if that damns business, then so be it. In fact, even if that damns culture, so be it. Or just much cultural enjoyment do you think you will have when we are all living in a privacy-less 1984-style world?
Thankfully though, we need not face so bleak a choice. We can have both culture, and the freedom to enjoy it. To be sure, we seem to be going in a direction where cultural proliferation is increasingly less likely to yield good ancillary businesses (think selling copies of stuff). But that proliferation, that development of our collective culture, will continue, there is every indication, long after these intellectual property things have become but mere footnotes in the annals of history.
So, to sum up, I indeed do not have the complete picture of how the post-copyright-et-al world will look like; but that is no excuse for us not to ditch the bloody mess.
PS: If the reader is about to object that most of the objections I put forth above only apply to copyright, I concede as much. This is because, I suspect, as he was talking about intellectual property, my colleague whom I refer to above must have been thinking about copyright. Yet another folly that results from using the redundant and misleading notion of intellectual property… (and if you’re wondering where the adjectives come from, see footnote #1).
This is in most cases simple to do; the exception might be when one of
the =
signs, usually the first one, needs to be invisible. The below
code solves that problem.
1 2 3 4 5 |
\begin{align*} &\phantom{{}={}}F(x\mid\mid x' + a\mid\mid a')\\ &=F\left [ (x+a) \mid\mid (x'+a')\right ]\\ &=f(x+a) \mid\mid f'(x'+a') \end{align*} |
The result is like so:
The difference between align
and align*
is that the latter
suppresses equation numbering.
To get, for instance, this:
you would do this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
\begin{displaymath} \begin{aligned} \begin{cases} x \equiv 56 \pmod{13} \\ x \equiv 33 \pmod{23} \\ x \equiv 45 \pmod{29} \end{cases} \end{aligned} \Leftrightarrow \begin{aligned} \begin{cases} x = 56 + 13 \alpha \\ x = 33 + 23 \beta \\ x = 45 + 29 \gamma \end{cases} \end{aligned} \end{displaymath} |
To cut a long story short, the aligned
environment is to be used when
you want a single equation number for several lines—which typically
happens when using the environment in the context of some larger
construction.
Note that inside the cases
environment, no &=
is required for
alignment—unlike what happens with align*
, shown above.
For spacing: In a “math” environment, LaTeX ignores the spaces you type and puts in the spacing that it thinks is best. LaTeX formats mathematics the way it’s done in mathematics texts. If you want different spacing, LaTeX provides the following four commands for use in math mode:
Sometime ago I wrote about to get MathJax working locally. Now here’s another thing you might need to do in MathJax: define new math commands. No worries:
]]>I use the projector
class for my presentations. Why? Glad you
asked. First, these days I’m using LaTeX for so many things, that it
seemed rather natural to use it also for presentations. Don’t get me
wrong, it is not without its drawbacks… for example being picky about
the exact layout of things, in a presentation done in LaTeX, might not
be the brightest of ideas… But my layouts tend to err on the side of
simplicity (more on this in a moment), and furthermore, the slides tend
to include quite a bit of mathematics – which is where LaTeX can really
excel. In what simplicity is concerned, I think most slides these days
contain WAY too much text:
The plain absurdity of modern academic talks would be glaring if we hadn’t all pickled in it for so long. Recall the last one you attended. The speaker flashed a slide full of words on the screen and talked. Did you read and understand the slide? Did you hear every word spoken? No, you had to make a decision about what to miss.
The slides should be something to help the audience understand what you are saying, rather then aesthetically pleasing graphical compositions, which only aid the audience in focusing away of what you are saying. Because of this simplicity, the precise layout ends up not being terribly important (you have less stuff on your slide, so you have more choices on how to arrange them). Or as the UNIX world saying goes, less is more!
Having made the case for using LaTeX, the elephant in the room is why I
decided to use an obscure template, instead of something like the
well-known beamer
class. projector
’s author, Victor Shoup, actually
has that one covered: «I didn’t feel like reading the beamer manual, so
I just made my own». As for me, well, projector
’s manual is several
orders of magnitude smaller than beamer
’s, so…
The basic skeleton of a slide is like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
\begin{slide} \Stripe{Slide 1} \vfill { \fadeFrom{2}{85} \item Lorem ipsum dolor sit amet } \vfill { \fadeTo{1}{85} \fadeFrom{3}{85} \item Consectetur adipiscing elit } \vfill { \fadeTo{2}{85} \item Sed do eiusmod tempor incididunt } \vfill \end{slide} |
The way this works is that it generates a slide, with the (imaginative…) title “Slide 1”, and with three bullet points. But these bullet points are not displayed all at once; rather they are shown in three overlays. In the first overlay, only the first bullet is shown “normally”; the other two are shown (in this case) with an 85% transparency. For the next overlay (in practice, the next “slide”), only item two is shown “normally”, whilst the first and third are shown with the transparency. The third overlay is similar, but only the third bullet is shown “normally”, i.e. with full colour.
The reason I do this, is to on the one hand, avoid distracting the reader with too much information (usually only one of the bullets is relevant at any given time), but on the other, to allow me to use the slide as a “memory card”: while an 85% transparency is sufficient to make the text essentially not visible—and hence not distracting—to the user, it still allows me, being much closer to the projection (usually), to be able to read that text, thus being able to keep track of what I have to say next.
So in essence, the above code for the slide shows one bullet at a time, from the first to the last.
This may seem very easy: just delete \fadeFrom{3}{85}
in the second
bullet. The problem is that if you do this, the program will have no
idea you want three overlays. It will use just two, which means the last
bullet will never be shown. You solve this by, in addition to removal of
the \fadeFrom
line, adding a \showAt
command:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
\begin{slide} \Stripe{Slide #1} \vfill { \fadeFrom{2}{85} \item Lorem ipsum dolor sit amet } \vfill { \fadeTo{1}{85} \item Consectetur adipiscing elit } \vfill { \fadeTo{2}{85} \showAt{3} \item Sed do eiusmod tempor incididunt } \vfill \end{slide} |
Now in the second overlay, only the second bullet will be shown; but on the third overlay, both the second and the third bullets will be shown.
To allow some variability in the type of bullets, I use some custom commands (that go in the preamble)…
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
% Different kinds of (symbol) bullets \newcommand{\bulletA}[1]{ % \hspace{0.5em}$\ast$ #1} \newcommand{\bulletB}[1]{ % \hspace{0.5em}$\bullet$ #1} \newcommand{\bulletD}[1]{ % \hspace{0.5em}$\diamond$ #1} \newcommand{\bulletT}[1]{ % \hspace{0.5em}$\triangleright$ #1} % Numbered bullet: % \bulletN{<number>} text... \newcommand{\bulletN}[2]{ % \hspace{0.5em}$#1.$ #2} \newcommand{\topic}[1]{ % \Banner{#1}} |
\bulletA
is an ∗
, \bulletB
is the regular round black-filled
bullet ∙
, \bulletD
is ⋄
, and \bulletT
is ▷
. Additionally,
there is also a numbered bullet, which takes its number as an argument.
As my presentation don’t have very long sequences, I doubt this will
ever become cumbersome.
Lastly there is the \topic
command, which simply uses projector
’s
\Banner
. I use it to group bullet by topic, as can be seen in the
example mentioned at the end of the post (the rename helps me remind of
the command I want).
I am not a fan of using those structures in presentations, but should
the need arise, you can do it as illustrated here (you need to define
the counter in the preamble \newcounter{tmpc}
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
\begin{slide} \Stripe{Spill into next slide} \begin{enumerate} \item Foo \vfill \item Continue enum into next slide \vfill \setcounter{tmpc}{\theenumi} \end{enumerate} \end{slide} \begin{slide} \Stripe{Enough already!} \begin{enumerate} \setcounter{enumi}{\thetmpc} \item Bar \vfill \item Baz \end{enumerate} \vfill \end{slide} |
projector
’s manual states that, for “TeXnical reasons” (ipsis
verbis), the verbatim
environment cannot be used inside a regular
slide
. Instead one has to use the slide*
environment. Here is an
example:
1 2 3 4 5 6 7 8 9 10 11 12 |
\begin{slide*} \Stripe{Verbathimgs} \vfill \bulletB Not verbatim text, vs.\ \verb+verbatim text+ \vfill \bulletB The \verb+verbatim+ \emph{environment} also works: \vfill \begin{verbatim} Lorem ipsur blah blah... \end{verbatim} \vfill \end{slide*} |
Dump the following in the preamble:
1 2 3 4 5 6 7 |
% math theorem environments \newtheoremframe{theorem}{Theorem} \newtheoremframe{corollary}{Corollary} \newtheoremframe{lemma}{Lemma} \newtheoremframe{definition}{Definition} % 'def' cannot be used as environ name \theoremstyle{remark} \newtheoremframe{remark}{Remark} |
Now to show, e.g. a theorem, do:
1 2 3 4 5 6 |
{ \fadeTo{3}{85} \begin{theorem}[(Euler's identity)] % Note the ( and ) in the optional name $e^{i\pi}-1=0$ \end{theorem} } |
As commented in the snippet, for the theorem/etc./’s name to be enclosed in parenthesis, these must written explicitly (unlike what happens with my “regular” LaTeX setup).
Say your image’s filename is photo.pdf
(other extensions might also
work; if not, imagemagick
is your friend). If it is a big image, that
is supposed to occupy most of the slide, you can do something like:
Otherwise, you can place that image in the slide like so:
1 2 |
\putbox{0.65\textwidth}{0.30\textheight} {\graphicbox[scale=0.19]{photo}} % Note: file extension unneeded |
\putbox
creates a box with at distance (in this case)
{0.65\textwidth}
and {0.30\textheight}
from the right and up of the
lower left corner of the slide, respectively. Adjust the scale
accordingly. (For more tricks with boxes, see section 5.8 of
projector
’s manual).
Most of presentation end with a big smiling question mark, occupying the
entire slide. It is drawn like shown below. The marvosym
package
(link), is needed for the \Smiley
symbol—I think it is already a part of TeXLive.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
% in preamble \usepackage{tikz} usepackage{marvosym} % for \Smiley % in document \begin{slide} \Stripe{Questions?} \vfill \begin{center} \begin{tikzpicture}[ultra thick,scale=0.5] \draw[line width=3.5,line cap=round] (1.5,0) .. controls ++(0,4) and ++(0,-2) .. (4,4) to[out=90,in=0] (2,6) to[out=180,in=90] (0,4); \end{tikzpicture} \end{center} \vfill \putbox{0.435\textwidth}{0.04\textheight}{\Huge \Smiley} \end{slide} |
For an example of this style of presentation, see this
page, file presentation.tex
and
presentation.pdf
. Some of the common preamble stuff (e.g. the new
commands) are in a separate file, inc_presentation_preamble.tex
.
Enjoy!
I am distressed to find that some women friends (fortunately not many) treat the use of the impersonal masculine pronoun as if it showed intention to exclude them. If there were any excluding to be done (happily there isn’t) I think I would sooner exclude men, but when I once tentatively tried referring to my abstract reader as ‘she’, a feminist denounced me for patronizing condescension: I ought to say ‘he-or-she’, and ‘his-or-her’. That is easy to do if you don’t care about language, but then if you don’t care about language you don’t deserve readers of either sex. Here, I have returned to the normal conventions of English pronouns. I may refer to the ‘reader’ as ‘he’, but I no more think of my readers as specifically male than a French speaker thinks of a table as female. As a matter of fact I believe I do, more often than not, think of my readers as female, but that is my personal affair and I’d hate to think that such considerations impinged on how I use my native language.
Although English is not my native language, this seems a sensible course of action, which is why I have adopted it.
]]>coderay
still does not support TeX highlighting, so
the output is in raw text. Hopefully it will not be too much of a
hindrance.
For “articles” for which you need a custom signature. Must be used in one-column mode only (and at the end of the body)!
1 2 3 4 5 6 7 8 |
\begin{minipage}[c]{\textwidth} \vspace{1cm} \flushright\parbox{7.5cm}{\emph{Yours sincerely}\\ \\[1.0cm] Your Truly's Full Name} \flushright\parbox{7.5cm}{} \vspace{1cm} \end{minipage} |
Done.
First, either in the preamble or in an *.sty
file, define the
following environment:
1 2 3 4 5 6 7 8 9 10 11 12 |
% custom dedication page \newenvironment{dedication}% { \clearpage % we want a new page \thispagestyle{empty} % no header and footer \vspace*{\stretch{1}} % some space at the top \itshape % the text is in italics \raggedleft % flush to the right margin }% { \par % end the paragraph \vspace{\stretch{3}} % space at bottom is three times that at the top \clearpage % finish off the page } |
I use the Lobster Two
font plus italics for the dedication; in Archlinux is comes with the
package texlive-fontsextra
; also remember I use
LuaLaTeX, so font commands for other
processors will likely differ. To use it, put in the preamble:
1 |
\newfontfamily\lobster{LobsterTwo} |
Then at the place where you want the dedication, put:
1 2 3 4 5 6 7 8 9 10 |
% Dedication %--------------------------------------------------------- { \fontfamily{LobsterTwo}\selectfont \lobster \begin{dedication} \large I dedicate this work to my TeX book's funny character, Dr. R. MaDillo, the giant\\ on whose shoulders a good TeX book has been written. \end{dedication}} \normalfont %--------------------------------------------------------- |
Done.
NOTE: The font can also be loaded by doing
\usepackage{LobsterTwo}
. However this will set that font as the
default roman one, for the entire document!! Beware!
The Epigraph package allows for «the pithy quotations often found at the start (or end) of a chapter». The problem is that I wanted to be able to set how much of the page’s width the quotation was suppose to take, for each quotation. The vanilla package does not allow this, so based on the ideas mentioned here, I came up with the following solution. In the preamble put:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
\usepackage{epigraph} \usepackage{calc} \newcommand{\mytextformat}{\epigraphsize\itshape} \newcommand{\mysourceformat}{\epigraphsize\scshape} \newenvironment{mytext}{\mytextformat}{} \newenvironment{mysource}{\mysourceformat\hfill}{} \renewcommand{\textflush}{flushleft} \renewcommand{\sourceflush}{flushright} \newlength{\epitextlen} \newlength{\episourcelen} \newlength{\epilength} \let\originalepigraph\epigraph % usage: \epigraph[<length>]{<quote>}{<author>} \renewcommand\epigraph[3][0.8]% { % \setlength{\epitextlen}{\widthof{\mytextformat#2}}% \setlength{\episourcelen}{\widthof{\mysourceformat#3}}% \ifdim\epitextlen>\episourcelen% \setlength{\epilength}{\epitextlen}% \else% \setlength{\epilength}{\episourcelen}% \fi% % \wlog{\epitextlen} % \wlog{\episourcelen} \ifdim\epilength>#1\textwidth% \setlength{\epilength}{#1\textwidth}% \fi% \setlength{\epigraphwidth}{\epilength}% \originalepigraph{\mytextformat#2}{\mysourceformat#3}% } |
The syntax of the new command now becomes
1 |
\epigraph[<length>]{<quote>}{<author>} |
On the optional length parameter: if omitted, the length of the epigraph will be that of either the quote or the author, whichever is largest—until the maximum of $80$% of the page’s width (the default). If not omitted, then it must be a number between $0$ and $1$, indicating the percentage of the page’s width which is to be used. Some examples:
1 2 3 4 5 |
\epigraph[0.7]{The whole point of cryptography is to solve problems. (Actually, that's the whole point of computers\emd something many people tend to forget.)}{Bruce Schneier} \epigraph{IRAV IVQV IVPV}{GAIVS IVLIVS C\AE SAR} |
Done.
1 2 3 4 5 |
\begin{figure}[ht] \centering \includegraphics[scale=1.5]{path/to/graphics/file} \caption{\label{fig:simplest}Simple graphic.} \end{figure} |
The [ht]
options to the environment instruct TeX to either place the
image at the place where the environment is, or, if that is not
possible, then placing it at the top of a page is preferred.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
\begin{table}[h] \centering \parbox{0.4\textwidth}{ \centering \begin{tabular}{c | c | c} + & 0 & 1 \\ \hline 0 & 0 & 1 \\ \hline 1 & 1 & 0 \\ \end{tabular} \caption{\label{tab:xor}Addition in GF2.}} \parbox{0.4\textwidth}{ \centering \begin{tabular}{c | c | c} \times & 0 & 1 \\ \hline 0 & 0 & 0 \\ \hline 1 & 0 & 1 \\ \end{tabular} \caption{\label{tab:xor}Multiplication in GF2.}} \end{table} |
Here each actual table is drawn by the tabular
environment; the
table
environment is actually the collection of all the tables in the
same “set”. The above snippet shows two tables, side by side
(horizontally). This is achieved by drawing each table (i.e. each
tabular
environment) inside a parbox
. Inside each of these there is
a \centering
command to have the table centred inside its parbox
(instead of left-aligned, the default). Finally to have the set of two
tables centred relative to the page, we use an extra \centering
outside of any parbox
(line $2$).
Endnotes are “footnotes” that are placed at the end of the document (instead of at the end of the package). Sometimes I use the two kinds of notes… so two distinguish them I set footnotes with Arabic numbers, and endnotes with roman numerals. Preamble:
Endnote example:
1 |
\endnote{This is an example endnote} |
Code to place at location where endnotes are to be displayed:
1 2 3 4 5 6 |
\begingroup \addcontentsline{toc}{section}{Notes} \renewcommand{\enotesize}{\small} %\def\enoteheading{\section*{Notas}} % if you need to change the Note's %section title \theendnotes \endgroup |
Done.
Self-explanatory feature provided by package marginnote. Preamble:
Example:
1 |
\marginnote{foobar this is a margin note} |
Done.
On certain type of documents, e.g. when describing a project or elaborating a proposal, you might want to have a section with references—but these will be very general, instead of the type that you cite for a specific fact or point. What I usually do in these cases is to have a “References” section, but one that besides merely listing the references, also has some accompanying text explaining why the references are relevant for the broader document.
It begins with—when using biblatex
—adding the relevant
bibliography file, which contains the references to be used, as is done
usually.
1 |
\addbibresource{article.bib} |
Next, add a numberless “References” section, with the explanatory text and the references per se, but without the automated title:
1 2 3 4 5 |
\section*{References} For the previous repertoire of songs, check the following references. \nocite{*} % can also be \nocite{key} \printbibliography[heading=none] |
If the bibliography file contains more entries than the ones you want
listed, you can use \nocite
with the relevant keys, to display just
those references (the *
causes all references in .bib
file to be
shown).
Done.
To reference, for instance, an online version of a book, stored in the
Gutenberg site, create in the project root a file named biblatex.cfg
,
with the following:
1 2 3 4 5 6 |
\DeclareFieldFormat{eprint:gutenberg}{% Project\space Gutenberg\space ebook\addcolon\space \ifhyperref {\href{http://www.gutenberg.org/ebooks/#1}{\nolinkurl{#1}}} {\nolinkurl{#1}}} \DeclareFieldAlias{eprint:GUTENBERG}{eprint:gutenberg} |
Now to add to the bibliography, for example, The Life and Letters of
Charles Darwin, put the following
in the .bib
file:
1 2 3 4 5 6 7 8 9 |
@book{Darwin_letters, title = "Life and Letters of Charles Darwin", author = "Charles Darwin", editor = "Francis Darwin", volume = "1", eprint = {2087}, eprinttype = {gutenberg}, year = "1887" } |
If you’re using hyperref
, a link to the above url will show up
automagically in the pdf. Profit! For more details, see
here.
When about to do a slight change of topic, I add some vertical space to mark the change, e.g.
The \noindent
is there because it makes no sense to indent a
paragraph if there already is vertical spacing separating it from the
previous one: the indenting serves precisely to mark that separation
without vertical spacing.
If the change of topic is more stark—but not enough so to justify a change in sectioning unit—then the asterisks are your friend:
1 2 3 4 5 6 |
Old paragraph. \noindent\raisebox{0pt}[3em][1em]{ \makebox[\textwidth][c]{*\hspace{1em}*\hspace{1em}*}}\\ \noindent New paragraph. |
Done.
]]>The basic principle, first put forth by Kerckhoffs, is that the safety of the cryptosystem rests solely on the key. That is, you have a plaintext message, you choose a key at “random” (more on this later), and obtain a ciphertext message; and the only way of obtaining the original plaintext from the ciphertext is to decrypt it with the same key.^{1}
First, in any “real world” scenario, the plaintexts are not all equally likely: some will be more likely than others (e.g. considering the set of all strings over the Latin alphabet, the plaintexts that correspond to (for instance) valid English will be more likely than those that are just a random sequence of characters). This means that there exists a probability distribution over the set of all possible plaintexts, which very likely is not the uniform distribution. Shannon calls this the a priori probability of a plaintext, and one must assume that this plaintext probability distribution is known to the cryptanalyst (remember we assume that the only thing the cryptanalyst does not know is the key—cf. above).
Now formally, a cryptosystem consists of three algorithms, . The first generates a random key, the second is the encryption algorithm that takes a plaintext and a key and outputs a ciphertext, and the third is the decryption algorithm that takes a ciphertext and a key and outputs a plaintext. A cryptosystem is consistent if for all and , with probability . The algorithm also implicitly defines a probability distribution over the set of possible keys, , and furthermore, one can always assume that that distribution is the uniform one.^{2} The probability distributions over and , together with the encryption algorithm , specify (again implicitly) the probability distribution over the set of possible ciphertexts, . The ensuing discussion assumes that the encryption algorithm is fixed.
Which brings us back to perfect secrecy. Shannon defined a cryptosystem as perfectly secret if, for every plaintext, the a priori probability was equal to the a posteriori probability; which is the probability that that plaintext was the one that originated the observed ciphertext. Mathematically we have the following
Definition 1 (Perfect secrecy). An encryption scheme is perfectly secret if for all and , and for all probability distributions over , we have ^{3}. End.
An elementary application of Bayes’ theorem shows that the previous definition is equivalent to having perfect secrecy if and only if (again for all , and probability distributions over ).
This is a rigorous definition, but when I first learned the concept, then read through the facts that stem from the definition, I felt some unease, due to the fact that for all the talk about probabilities, their space was never defined. So let’s do that now. This set is composed of all the triples , where , and . There are such tuples. The probability of is , because when the encryption algorithm is fixed, is fully determined by the plaintext and the key. Note that , because and , and multiplying both yields the initial summation.
This may seem as a pointless display of pedantry, but its value becomes obvious when one tries to understand (and calculate) probabilities like , where is a fixed ciphertext. (A remark about notation: values that are assumed fixed are not subscripted.) Intuitively, one could surmise it should be something like the summation of the probabilities for all possible plaintexts, each multiplied by the probability of choosing a key that encrypts that plaintext into . The formalism of the previous paragraph allows us to verify this conjecture. Indeed, to calculate , just select all tuples where , and sum their probability. We obtain
To see how this is equivalent to our intuitive guess, consider what happens if for a given , there are two different keys (say and ) that encrypt it to . Then we would have:
So we conclude that each is multiplied by the total probability of selecting a key that encrypts it to —just as conjectured.
Both forms of Definition 1 are based on conditional probabilities. Let’s see what insight our formalism can provide on those events.
We can re-write the second summation in Equation 1 differently, by noting that each of the terms is just the probability that both and occur:
It is implicit that this is done for all keys that encrypt into . This makes sense because the plaintexts form a partition of the probability space: (which is the same as saying that the sum of the probabilities for all plaintexts is ). On the other hand, for a given , that is encrypted into by keys and ,
This is because is the sum of the probabilities of the tuples of the form , and in our example there are two such tuples, and summing their probabilities yields . Dividing by we get . This of course holds for more than two keys. But this means this sum is also equal to , and this in turn allows us to, yet again, re-write the probability like so:
As far as I can tell, there is no description of that is similar to Equation 2, because this depends on the probability distribution of the plaintext. The best we can write for is the following, which is not simple at all…
Despite all the talk about perfectly secret ciphers, the truth is that so far all the equalities shown are valid for any symmetric encryption scheme (not just for perfectly secret ones). The next two however, are only true if the cipher has perfect secrecy. First using conditional probabilities we prove another equivalent condition to (both forms of) Definition 1.
Theorem 1. A cipher has perfect secrecy, iff for any distribution over , any and any , we have . End.
Proof. () If the cipher has perfect secrecy, then $P(e|m_1)=P(e)=P(e|m_2)m_1m_2e$$.
() If , for any , and , then
$P(e|m)P(e|m_i)m_i$$, because it is always the same by hypothesis. QED.
Next we prove another necessary and sufficient condition for perfect secrecy. In the case of such a cipher, has always the same value, for all (viz. ). And because we can always assume that the keys are generated according to the uniform distribution, this means that, for a fixed , and for any , the number of keys that encrypt to is always the same. The next result shows the converse is also true.
Theorem 2. A cipher has perfect secrecy if and only if, having fixed a ciphertext, for any plaintext, the number of keys that encrypt it to that fixed ciphertext is the same (but note this number can vary for different ciphertexts). More formally, let be a fixed ciphertext as before, and let be the set of keys that encrypt to . Then a cipher has perfect secrecy iff has the same value, for all .^{4} End.
Note that the analog property when having fixed a plaintext is false: there can be more keys that encrypt that plaintext to one ciphertext than to another ciphertext, but the cipher can still be perfectly secret. We’ll see an example of this shortly.
Proof. () We have argued this direction informally, based on the property that, for perfectly secret ciphers, . But we can also use . From (3), if the cipher is perfectly secret:
is constant in the numerator summation, so it can be put outside it, and cancelled with the of the right hand side. We thus obtain
Remember the denominator equals . What this means is that for a given (fixed) ciphertext , and for all plaintext messages , the number of keys that encrypt to must have probabilities that always sum to the same value. Given the assumption of a uniform key distribution, this is the same as saying the number of keys must always be the same.
() If for any ciphertext , the number of keys that decrypt it to any plaintext is the same, then we immediately have that for any two different plaintext messages, and , it must be the case that . Theorem 1 now yields the conclusion that the cipher is perfectly secure. QED.
After that lengthy introduction, we now finally come to the question that actually annoyed me enough to try to visualise the probability space in the way I’ve just described. That question is the following: does perfect secrecy imply the ciphertext distribution is uniform? I could not either prove it or refute it, but it turns out the answer is no. Here’s the counterexample: we have two bits of plaintext, , four bits of key material , and three bits of ciphertext :
This encryption algorithm has perfect secrecy, because for any given ciphertext, there is the same number of keys that decrypt it to any plaintext (cf. Theorem 2). This is straightforward (if somewhat laborious) to see.
Consider the ciphertext , and an arbitrary plaintext . What are the keys that would encrypt the said plaintext into the said ciphertext? Given that must be zero, we have that . The same reasoning yields , because . Finally, given that and , it must be the case that . This yields the following three possible keys for encrypting into : , and . We denote this set as .
Reasoning similarly, we can write the following table, listing the keys that encrypt an arbitrary plaintext into the ciphertext in the left column. An overline () denotes the complement of the bit .
Ciphertext | Keys |
---|---|
$$(0, 0, 0)$$ | $$(b_0, b_1, \{(0, 0), (0, 1), (1, 0)\})$$ |
$$(0, 0, 1)$$ | $$(b_0, b_1, 1, 1)$$ |
$$(0, 1, 0)$$ | $$(b_0, \overline{b_1}, \{(0, 0), (0, 1), (1, 0)\})$$ |
$$(0, 1, 1)$$ | $$(\overline{b_0}, \overline{b_1}, 1, 1)$$ |
$$(1, 0, 0)$$ | $$(b_0, b_1, \{(0, 0), (0, 1), (1, 0)\})$$ |
$$(1, 0, 1)$$ | $$(\overline{b_0}, b_1, \{(0, 0), (0, 1), (1, 0)\})$$ |
$$(1, 1, 0)$$ | $$(b_0, \overline{b_1}, 1, 1)$$ |
$$(1, 1, 1)$$ | $$(\overline{b_0}, \overline{b_1}, \{(0, 0), (0, 1), (1, 0)\})$$ |
Thus we can see that for any fixed ciphertext, there is the same number of keys that cause it to decrypt to any plaintext; thus the scheme has perfect secrecy. However, the ciphertext distribution is not always uniform: if both plaintext and keys are assumed uniform, then (for example) the ciphertext will be more likely to appear than , because there are more keys that encrypt an arbitrary plaintext to it. In other words, although the cipher is perfectly secret for all plaintext distributions, there is at least one (viz. the uniform distribution) for which the ciphertext distribution will not be uniform.
Also recall the remark made after stating Theorem 2: in this cipher for a given plaintext, there are more keys that encrypt it to some ciphertexts rather than others—indeed that is the cause of the non-uniformity of the ciphertexts—but it does not prevent perfect secrecy, as this example illustrates.
And what about the converse? I.e. if the ciphertext is uniformly distributed, then is the cipher perfectly secret? As already mentioned, the ciphertext distribution is implicitly specified by the encryption algorithm, and key and plaintext distributions. Given that we can assume the key to be uniformly distributed, if the ciphertext is also uniformly distributed we have (notice that the first summation is just another notation for writing (1)):
Remember that is the set of keys that encrypt to , and is the summation of the probabilities of those keys. If the ciphertext is uniformly distributed regardless of the plaintext distribution, then we must have . This is only possible if, having fixed an , as the same value for all —we denote that common value by (notice the subscript is gone). The uniformity of the key distribution now means for a fixed , has the same value, for all . Theorem 2 now yields that the cipher is perfectly secret. Thus we can now state
Theorem 3. If a cipher has an uniform ciphertext distribution, regardless of the plaintext distribution, then it is a perfectly secret cipher. End.
The converse is false, as the above example cipher shows. An example of an encryption scheme where the ciphertext distribution is always uniform, is the One Time Pad.
Notice that in this case—uniform ciphertext distribution—as mentioned above, fixing , the number of keys that encrypt to is the same, for all . But in addition to that, because has the same value for all , so does . This means that for a fixed plaintext , we have has the same value, for all . Or in words, for all plaintexts, the number of keys that encrypt any particular plaintext to any particular ciphertext is the same.
This assumes that given one specific plaintext and ciphertext, there is only key that encrypts the former into the latter (and vice-versa for decryption). This need not be the case (even for perfect secrecy!), as we shall see. But of course, for any secure cryptosystem, even if they do exist, it should be infeasible to calculate one such key. ↩
Consider a cryptosystem , in which is a really complicated algorithm, that outputs the key according to a very non-uniform distribution. This algorithm usually takes as input a “random tape”, which outputs symbols of the same alphabet, according to a uniform distribution. Then we can use this random tape as a new key generation algorithm , and incorporate the operations of in and . Thus we obtain a derived cryptosystem , with a uniformly generated key, but with the same ciphertext distribution—because the operations are still the same, they are just “in a different place”.
More concretely, consider a specific tuple , with probability . Let be the (uniformly random) key which used as its random tape to produce key . This means that , where the last term is the probability that produced key when its random tape produced . Then, under the new cryptosystem, what is the probability of ? Remember that the previous has been incorporated into , which means that the probability is that of being selected, times the probability of being selected, times the probability of internally transforming into . I.e. ; but this is just the original value. In particular this means that for any and , the value of does not change (cf. footnote #4).
As an ending remark, note that if the only source of randomness for is the random tape, then , i.e. is always transformed in . ↩
To be rigorous, we would have to define random variables and , and say that , where and . But such a level of rigour is not needed here. ↩
To better illustrate what the assumption of uniformly generated keys means in this context, suppose that (for some perfectly secure cryptosystem), for a given ciphertext , there exists a plaintext , for which there are two keys that encrypt it to , and a plaintext for which there are three such keys. Furthermore, suppose that the key distribution is such that . By the method of footnote #2, if we now produce an equivalent cryptosystem with uniform keys, the ciphertext distribution does not change, so neither do or ; but they can no longer be equal, because as there are more keys for than , we must have . This shows the original cipher could not have been perfectly secure. ↩
flickr
. It was good, and I stuck
with it—until I logged in from abroad, and got locked out because I
could not answer a phone challenge, considering how I had not given them
my phone number (nor having any intention to do so).
So I was back at having my photos only in my hard drive, until I heard
of Google Photo
. It’s free, unlimited, and it totally lacks privacy
(from Google), but for the photos I intend to store there, it’s more
than enough. What I had stored in flickr
encompasses roughly four
categories; here are the corresponding albums @ google:
Now one of the problems is that (unlike flickr
) I can’t list all of
the albums (or all of the photos, for that matter). But I can route
around that problem. As Prof. Fred Brooks pointed
out, one must
«[p]resent to inform, not to impress;», because «if you inform, you
will impress.». So while I cannot dump all of the photos and provide a
link for you to sift through, I can put the relevant new ones in
(public) albums, and write a blog post embedding them, thus making them
part of the story. And besides, that way I get to debut a new category!
;-)
OK, so having ArchLinux already pre-installed in my laptop, I decided that the best thing to do is install Debian from Arch. What I describe next are the changes to that procedure that I have to do to get it to work.
So first, this method of installation requires the debootstrap
tool;
for Archers, you just have to get it from the AUR.
Next, after creating the chroot
environment and chroot
-ing into it,
the PATH environment variable will not be set—which means that to run
commands at the prompt, you’ll have to invoke them with their full
path—not very practical. The solution is to set PATH to the value of
ENV_SUPATH
in the /etc/login.defs
file.
1 |
# export PATH=<value from `/etc/login.defs`> |
Next, when setting the timezone, there exists nothing in the /etc/rcS
file that is used to indicate «whether the system will interpret the
hardware clock as being set to UTC or local time». But this did not
seemed to cause trouble so, moving on…
Next, I stumbled across one weird error when running aptitude
update
. It complained that some signatures could not be verified, but
it turned out that it was caused by a network outage I had failed to
notice…
Next on the menu is dealing with the bootloader. I use GRUB, so it was
just a matter of adding an entry for Debian. The partition where Deb is
installed is /dev/sda4
, so in file /etc/grub/40_custom
I appended
the following:
After that you have to run # grub-config -o /boot/grub/grub.cfg
to
generate the GRUB configuration with the new entry. In the snippet
above, the reason you have to specify the root location twice is that in
more advanced setups, the values could be different: in the first line
we tell GRUB where to find the linux image (vmlinuz
), whilst in the
second one we tell the linux kernel where the root filesystem is
located. For example if we had set up a separate boot partition, those
values would differ. In the second line you can also add kernel options;
the ones I used were taken from here.
A quick side note about initrd
: in my first attempt, the Debian system
I ended up with was not bootable. I thought at first that this might be
because the initrd
was missing a module for ext4
, the filesystem I
choose to use. So I added the module, like so:
Next, we have to deal with users and passwords. The first step is to set
root
’s password: just run passwd
. Then you have to add an everyday,
non-root user: run adduser
. Next you have to install and set up
sudo
, and KDE.
About KDE, when selecting the keyboard layout, two of the choices were “Português” and “Português nativo”. DO NOT CHOOSE THE LATTER! It completely screwed up my keyboard, inside KDE, to the point I was not able to login (because I could not type the password). Always choose “Português”! Anyway, that ordeal forced me to reconfigure the keyboard, which is done like so:
1 |
# dpkg-reconfigure keyboard-configuration |
For testing, I also required the startx
command (to avoid having to
reboot the machine to see if my latest tweak solved the problem…). It
is in the xorg
package.
Finally, although I only noticed this after installing Skype, sound was not working in Debian. To fix this, do in root, and reboot:
Actually, this got sound working on Debian generally—but not on Skype. More on that below.
The instructions for amd64 in Debian’s wiki are sufficient:
1 2 3 4 5 |
# dpkg --add-architecture i386 # apt-get update # wget -O skype-install.deb http://www.skype.com/go/getskype-linux-deb # dpkg -i skype-install.deb # apt-get -f install |
The first line enables Multiverse (remember Skype is 32bit only, whilst we are now in a 64bit Debian…), and the last fixes missing dependencies. Beware to check any packages that last command might want to remove, for some of those may be potentially still necessary!
Now, as mentioned before, there was no audio in Skype, and it turns out
the fine wiki has the answer for this as well: Skype now requires
pulseaudio
to be installed. The Debian installer already installed the
libraries for this; I was only missing the server, so:
1 |
# aptitude install pulseaudio |
Notice that this will start the pulseaudio server, and set it up for starting when logging in.
I end this post by noting that if you use an external webcam and/or microphone (as you should), these have to be selected in Skype after been plugged in.
That’s it! Enjoy your conversations and, while you’re at it, don’t
forget to thank the patriotic men and women who have nothing better
to do tirelessly monitor everything we do and say, to stop The Bad
Guys™ from triumphing! Have fun!
So first we have to distill meanings: trust means trust in a key’s owner: i.e. how carefully will he check other people’s keys before signing them. Validity is a measure of the trust you have that a certain key does belong to its purported owner. Trust has the following four levels:
Now, as far as I can understand it, the same four levels apply to validity. So you can have a key that has full validity, marginal validity, or unknown validity—but I have no idea of what a validity of none means.
The other important bit you need to realise is that trust is set by the user, whilst validity is automatically determined by GnuPG. The rules for determining it seem to be as follows:
This last step means that there has to exist a sequence of keys like so: you key signs some key A, that in turn signs other key B, … until some key that signs the target key—where the path contains six or less keys (including the target and your own)—i.e. the path length is five or less.
This seems to explain the results in the example scenario outlined in the above linked section of GnuPG’s user manual. I hope it helps…
For some reason I was overly enthusiastic when I wrote that… Yes GnuPG sucks, although by the looks of it the manual manages to suck even more… At any rate, as Mathew Green pointed out, «a PGP critic is just a PGP user who’s actually used the software for a while», so… ↩
gpg -K
produced not output at all! With (a lot of)
hindsight, this was probably caused by two factors: one, the change of
format for the storage of secret keys in GPG 2.1, and the fact that
the ~/.gnupg/private-keys-v1.d
directory, where secret keys are now
kept, had no executable permissions, which means its contents were
inaccessible to every user or application! Now how did that came to
happen, I haven’t got the foggiest. But by the time I realised this as a
likely cause for my predicament, I was already well on my way to solving
the problem through the hard approach—which is what I now describe,
should I ever come to need it again…
OK, so remember that my “everyday” ~/.gnupg
does not contain my
private master key, so I keep a backup of my “full keypair” in, lets
say, /media/Secure/dotgnupg
. What I did was basically export from my
full keypair those parts that had gone missing in my everyday
~/.gnupg
, and import then therein. So, to the code, that talking (or
in this case writing…) is cheap!
Back up your stuff, and generate a new dummy key (to initialise the new
~/.gnupg
), which we’ll get rid of at the end.
Then, export the missing subkeys, and re-import then in the everyday
.~/.gnupg
. Note that this will also export any expired (and, I assume,
revoked) subkeys, thus allowing you decrypt and verify signatures on
older messages.
1 2 |
gpg --homedir /media/Secure/dotgnupg --export-secret-subkeys MYKEYID > my.subkeys gpg --import my.subkeys |
If you now list your secret keys, the only thing that should be missing
(i.e. with an #
in front) is the master private key.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
$ gpg --list-secret-keys --list-options show-unusable-subkeys /home/vrael/.gnupg/pubring.kbx ------------------------------ sec rsa2048/81FDCED3 2014-12-15 uid [ultimate] bilbo <bilbo@foo.org> ssb rsa2048/F5A10CA6 2014-12-15 sec# rsa4096/B58761E3 2013-02-09 uid [ unknown] Óscar Pereira (Check https://blog.erroneousthoughts.org/pgp-keys/ for up-to-date key status info) <oscar@erroneousthoughts.org> uid [ unknown] Óscar Pereira <burn.till.skid@gmail.com> ssb rsa2048/BAB2746E 2013-02-09 [expired: 2014-02-09] ssb elg2048/A79BBD4C 2013-02-09 [expired: 2014-02-09] ssb rsa4096/2D196E9F 2014-02-23 [expires: 2019-02-22] ssb rsa4096/BA911BB2 2014-02-23 [expires: 2019-02-22] |
Now we have but to delete the dummy keypair (and the backup of
~/.gnupg
)^{1}:
Notice that having been just freshly imported, your keys (or more
precisely your uids) have unknown
validity—which is of course
nonsense. To set them to ultimate
do (and repeat it for uid 2):
1 |
$ gpg --edit-key B58761E3 / uid 1 / trust / 5 /save / quit |
And we are done. Two endnotes:
Before unmounting /media/Secure/dotgnupg
, you need to killall
gpg-agent
, otherwise you get an error message saying that the
device is busy… (also note you don’t have to restart the agent
afterwords, because now it is started by gpg
whenever needed).
If you have a lot of signatures in your key, Kgpg
has a nice
feature: which is to automatically import all the keys corresponding
to those signatures.
I won’t deal with it here, but if you have a custom
gpg-agent.conf
file, you might also want to copy it from the
backup before deletion. ↩
$\varphi$ | $\psi$ | $\varphi \rightarrow \psi$ |
T | T | T |
T | F | F |
F | T | T |
F | F | T |
The first two lines are easy to understand, but the problem begins when one tries to make sense of the last two. Why can’t they be false, instead of true?
To make sense of it, one has to take a step back and think about what does it mean to treat “implication” as a binary function. So imagine you have two statements, $\varphi$ and $\psi$. Treating implication as a binary function essentially means saying “$\varphi$ might imply $\psi$”, depending on the values of $\varphi$ and $\psi$! I.e. the value of $\varphi \rightarrow \psi$ might be true of false, depending on the values of $\varphi$ and $\psi$! This is not how one is intuitively thinks about implication.
In fact, in common language one usually says that $\varphi$ implies $\psi$ to mean that there exists some “external” relationship between the two statements such that, if $\varphi$ is true, then that will cause for $\psi$ to be true as well. That “external” relationship is what we usually think of as causality. And that is a problematic notion, because it steers us away from mathematics and into the realm of philosophers. For example, if I shoot someone, and that person dies on the spot, there is clear causal relationship. But if I shoot someone, say, in the leg, with depleted uranium ammunition, and five years later that person dies of cancer, is there still a causal relationship? And where does one draw the “boundary of causality” (lacking a better name..)?
So you see that causality is just too much trouble so the mathematicians, being the lazy wise-arses that they are, decided to just get rid of it altogether, and replace causality with a mechanical rule, viz. the usual truth table of the implication, shown above. But they did in such a way as to be (very) useful! What to I mean by this? That despite being a mechanical rule, whenever there happens to exist a causal relation between $\varphi$ and $\psi$ (by whatever means that is ascertained), then $\varphi \rightarrow \psi$ will be true! But how is this achieved, without being able to know when there exists causality and when there does not? Well, the trick is that although it is usually very difficult to determine when there exists causality, it is very easy to determine when there cannot exist a causal relation: if the antecedent is true, but the consequent false. In other words, no causal relation can exist when the “cause” failed to produce its effect (the “consequence”). In symbols, if $\varphi$ is true and $\psi$ is false, then $\varphi \rightarrow \psi$ must be false. Thus we have explained the second row on the truth table. Question is, what do we do in the other three possible cases? Well, in these scenarios, there might exist a causal relation between $\varphi$ and $\psi$, or there might not. In order to guarantee that our mechanical rule is always true when there exists causality, we just define it to be true in the remaining three possibilities.
If this sounds like cheating, I sympathise. But the thing to remember is that the logical connective called implication we have just defined is not about causality—it’s just a binary function. So why is this useful? Because in mathematics, when we derive one statement from another, we have a causal relation of sorts: accepting the truth of the antecedent forces the acceptance of the truth of the consequence. And as we’ve just seen, when that is the case, our binary function behaves just as we expect it to^{2}.
To sum it up, the logical connective “implication” is designed as a mechanic rule, that nonetheless behaves as a “true implication” (i.e. as one would expect) when there is indeed a causal relation between the antecedent $\varphi$ and the consequent $\psi$. Not a small achievement, if you think about it.
EDIT: One way of testing if the reader has grasped the way the term “implies” is used in mathematics, is to try to solve this “paradox”.
EDIT 2: Added some clarifying remarks to the last paragraph.
Note, in particular, that if there is a causal relation between $\varphi$ and $\psi$, than that causal relation remains true even if $\varphi$ is false. For example, if “I drink poison” will make the assertion “I will die” true, that causal relation continues to be true, even if I refrain from ingesting poisonous beverages (i.e. if the antecedent is false). ↩
1 2 3 4 |
$ git pull origin master # Get the latest Octopress $ bundle update # Keep gems updated $ rake update_source # update the template's source $ rake update_style # update the template's style |
Next, it seems for some time now, setting published:
false
to your post causes Octopress’ preview ignore it. No matter, just
add Actually, you need to edit your unpublished: true
to your _config.yml
and all is nice
again.Rakefile
as per the
instructions of rcmdnk, given here.
EDIT: Another issue, though not exactly with Octopress, is that if you
use Stchangg’s mathy theme, then in order to get site searching
using google to work, you need to change
source/_includes/navigation.html
as described here.
EDIT 2: Still about the mathy theme, to get the date with no time, edit
source/_includes/post/date.html
and replace
1 2 3 4 5 |
{% if page.date %}{% capture time %}{{ page.date_time_html }}{% endcapture %}{% endif %} {% if post.date %}{% capture time %}{{ post.date_time_html }}{% endcapture %}{% endif %} {% if page.updated %}{% capture updated %}{{ page.date_time_updated_html }}{% endcapture %}{% endif %} {% if post.updated %}{% capture updated %}{{ post.date_time_updated_html }}{% endcapture %}{% endif %} |
with
1 2 3 4 5 |
{% if page.date %}{% capture time %}{{ page.date_html }}{% endcapture %}{% endif %} {% if post.date %}{% capture time %}{{ post.date_html }}{% endcapture %}{% endif %} {% if page.updated %}{% capture updated %}{{ page.date_updated_html }}{% endcapture %}{% endif %} {% if post.updated %}{% capture updated %}{{ post.date_updated_html }}{% endcapture %}{% endif %} |
Note that this will be reverted after updating the theme, so you might have to re-do this change afterwords…
EDIT 3: As I wrote EDIT 2 above, I had to post a snippet which contains
Liquid tags. Ordinarily these would be processed, and thus not be
shown. To avoid that, put the snippet lines between raw
and endraw
tags, like shown below. That way, Liquid tags between those two will be
ignored.
EDIT 4: When doing a rake preview
, Octopress will watch for changes in
the blog directory, but not (necessarily) in the other pages… And you
get a warning about this, telling you to use the --watch
option if
want Octopress to do so. Well, this also wrong. To preview changes in
pages, use rake preview watch
.
OK, ingredients: Linux, MathJax, ligHTTPd, Javascript enabled browser. Recipe:
MathJax.js
is in the /path/to/mathjax/
directory.ligHTTPd
. For ArchLinux
, instructions are here.Create ligHTTPd
’s configuration file, with the following contents:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
server.port = 8000 server.bind = "127.0.0.1" server.username = "http" server.groupname = "http" server.document-root = "/path/to/mathjax" server.errorlog = "/path/to/mathjax/error.log" dir-listing.activate = "enable" index-file.names = ( "index.html" ) mimetype.assign = ( ".html" => "text/html", ".txt" => "text/plain", ".css" => "text/css", ".js" => "application/x-javascript", ".jpg" => "image/jpeg", ".jpeg" => "image/jpeg", ".gif" => "image/gif", ".png" => "image/png", "" => "application/octet-stream" ) server.modules += ( "mod_setenv" ) setenv.add-response-header = ( "Access-Control-Allow-Origin" => "*" ) |
Replace the document-root
and errorlog
paths according to your set
up. The last line ensures that the browsers will not complain about the
script coming from a different origin.
Test your configuration running:
Run the server like so:
1 |
$ lighttpd -D -f /etc/lighttpd/lighttpd.conf |
The -D
ensures that the server won’t go into background (decreases the
chances you will forget about it and leave an unneeded server running).
In whatever application you need to have math rendered, use the following as MathJax’s location:
1 |
http://localhost:8000/MathJax.js?config=TeX-AMS_HTML |
Done!
EXTRA: If you want an awesome MathJax config, in
/path/to/mathjax/config/local/
create a file named
MathJax-local-config.js
, with the contents below, and invoke it like
so:
1 |
http://localhost:8000/MathJax.js?config=TeX-AMS_HTML,local/MathJax-local-config |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
MathJax.Hub.Config({ tex2jax: { inlineMath: [ ['$', '$'], ['\\(', '\\)'] ], displayMath: [ ['$$', '$$'], ['\\[', '\\]']], processEscapes: true, skipTags: ['script', 'noscript', 'style', 'textarea', 'pre', 'code'] }, "HTML-CSS": { scale: 105, styles: { "mathjax#MathJax_Zoom": { "background-color": "#a0a0a0" } }, }, menuSettings: { zoom: "Click", zscale: "170%", } }); MathJax.Ajax.loadComplete("[MathJax]/config/local/MathJax-local-config.js"); |
Next step: enjoy the fruits of your labour!
]]>This is all well and good for counting animals, but after a while,
someone invented this thing called writing, and now you must
make some symbols to indicate how big is your livestock. Pebbles
won’t be of any help, but you can replace them with an arbitrary
mark: for simplicity, lets use the vertical dash. Alas, you’ve
discovered the unary number system: |
, ||
, |||
, and so on
and so forth. Eventually you realise that those sticks quickly
become too numerous to count, so after some time, you invent some
new symbols for pre-specified quantities, and some elementary rules
to avoid ambiguities. And for centuries, this was the adopted
solution, albeit in different guises: we have the Roman numerals,
the Greek numerals, etc.
And again all was well and good, until someone invented agriculture. Now, besides having to eat vegetables, you have to calculate areas (of farmable land), which require multiplications, which are really hard to do with the existing number systems. Those systems are (reasonably…) adequate for addition, but multiplication, that’s a whole new ball game.
And that’s where one of the greatest inventions in mathematics comes into play: the zero. It simplified all arithmetic operations, in particular multiplication. But its novelty was deeper than it might seem: until the zero was thought up, all the other number systems had symbols for specific quantities. With the zero, came the positional number system, in which a given symbol could represent different quantities, depending on the position in which it was found. But for all I could research, I could not find the rationale that would lead to the appearance of the zero, so I (rather humbly), decided to invent my own. As usual, details are below the fold.
So, how shall we invent the zero? Well, that’s the wrong question to
ask. “Counting” means associating a different symbol with every
possible number of elements. “Elements” here means the units of
whatever it is you wish to count: vertical bars (|
), or apples, or
rocks or, of course, good old cows. “Symbol” refers to the marks
that are associated with a given number of elements: here’s we’ll
use Greek letters (see next paragraph). The correspondence between
the symbols and the possible number of elements must be
one-to-one: for a given number of elements, there must be only one
symbol, and one symbol must correspond to only one specific number
of elements. What we want is a set of symbols and rules that allows
us to re-use them in order to match any possible number of
elements. Given that we will only have a finite number of symbols,
at some point we will have to start re-using, i.e. we will need a
set of symbols (as opposed to just one individual symbol).
Now, we need some symbols, to begin with, so to avoid over-complicating things, lets work with just three^{1}: $\alpha$, $\beta$ and $\gamma$, with the following meanings: $\alpha \rightarrow |$, $\beta \rightarrow ||$ and $\gamma \rightarrow |||$. Now suppose you have a set of 4 things, but you want to avoid adding new symbols, if possible (hereinafter, when I use the usual hindu-arabic numerals, it’s just as a shorthand: you should think of them as representing a concrete set of… something. So ‘4’ might mean 4 apples, or 4 pebbles, or $||||$, or$\ldots$). OK, so you have a set of 4 elements, and only 3 symbols. What do you do? You start re-using the symbols.
After some neuron munching, you come up with a concrete approach: group things in sets of 3, until you only have less than 3 things out of any set. Why 3? Well, it can’t be 5 or bigger, because then, how would you represent 4? You can’t form any groups yet, but you also don’t have enough symbols for 4. And from the other end, it does not make sense to have groups of size 2 either, because for 3 you would have two representations: either with one symbol, , or with two: one for the set of two elements, and one for the single element out of that set^{2}. So we’re left with the choices of 3 and 4 for the size of the group. I don’t see any reason to choose one over the other, so lets go with Occam’s razor, and use the one that seems to be the simplest: the one where the size of the group matches the number of symbols. So groups of size 3 it is.
The first case where we need a set of symbols to represent a quantity, is 4. As discussed above, we need to represent a group of 3 elements, and 1 element outside that group. Given that we only know the number of elements outside any group after knowing the number of groups, it makes sense to represent 4 as —the first represents the 1 set of 3 elements, and the second one the 1 element out of any set. And what about 5? : one set of 3 elements and 2 elements out of that set. And 6? Well that’s… uh oh. Now you have two sets of 3 elements, and no elements outside of any set. Or in other words, you have zero elements outside any set, but we don’t have a symbol for that. And we don’t want to add any more symbols—that’s why we’re going to all this trouble to begin with! But wait a minute, this does seem a rather unusual situation, so maybe it does warrant its own symbol. OK, lets go with that hypothesis for the moment, and use the symbol to designate the fact that there are no elements outside of any set.
NOTE: in the above scheme, the order of the symbols is relevant! Thus we are no longer dealing with sets, but with sequences, or strings of symbols. (But we continue to reason in terms of sets of the elements which are being counted—for here the order is still irrelevant).
Well, now we can write 6 as : two sets of 3 elements each, and no element out of those sets! What about 7? . 8? . 9? : three sets of 3 elements each, and no element outside any set. So far so good, but what about 10? We’ve ran out of symbols again! Well, what did we do last time we ran out of symbols? We grouped elements together. But this time we have no elements (out of sets), only sets. So… can we group those sets? Lacking other ideas, lets do that: so we now have a set with 3 (sub-)sets, each with 3 elements. (Phew!) So this new superset of ours has 9 elements. And as there are no (3 element) sets outside that superset, and no elements outside any (simple-)set, we can now write 9 as . But wait a minute, something is wrong! Indeed, we now have two different ways of writing 9: and ! What’s going on here??
The problem is that we’re building our biggest sets by grouping 3 of the previous biggest sets together, but we also have a symbol for representing 3 of those previous biggest sets without any grouping: . This problem actually shows up way before 9, but we missed it: remember that 3 is represented with , but 3 is also a set of 3 elements! Which means that we could represent 3 like ! Or put another way: . But lo and behold: we now have a golden opportunity to go back to using only 3 symbols! So lets ditch , and adopt for “no elements”, or “zero”, and see where that leads us.
Notice that the only quantities which get different representations are (what we today call) the powers of 3: becomes as we’ve seen, and goes from to . For these are the only cases where the double-representation problem arises; everything else remains the same. Also notice that we have gained a simple way of expressing the sizes of our sets: the smallest size is (that takes just 1 element), the next one is , that takes 3 elements, the next is , that takes 9, and so on and so forth.
Having analysed the case with groups of 3, it’s worth to have a look at the case of 4. So 1 is , 2 is , and 3 is ; what is 4? We have no symbols for element number four, but we can look at it as one group of 4 elements… and no element outside that group. We’ve encountered this situation before, and as we did then, lets use the symbol for that case: 4 now becomes . 5 is , etc., 7 is , 8 is , …, 12 is , …, 15 is . And what about 16? For that we need more than 3 groups of 4 elements: in fact, we need 4 groups of 4! How can we represent this? We could use two symbols for the number of groups of 4. Thus, 16 would be , where the rightmost is for the number of elements out of any group, and the remaining represents 4 groups of 4 elements. But lo and behold, this is the same result than if we added another group, of 4 sets of 4 elements each! This is strikingly similar to what happened with groups of 3! Hum… we’re on to something here…
To avoid getting lost in our own reasoning, lets review what we have so far. We want to be able to count things, but using only a fixed set of symbols. To achieve that, we thought up a way of using those symbols to produce a set of sequences of symbols which can than be put in one-to-one correspondence with the elements being counted. Actually that’s not true, we thought up two ways of doing that: aggregating the elements in groups of 3, and in groups of 4. Both provide a way to build unique sequences of symbols, as long as we like. Thus both can be used for counting (albeit the same sequence will have different meanings in the two systems).
Basing our system in groups of 3 requires 3 symbols; groups of 4 require 4 symbols. But there’s never a symbol for the number of elements in the group; rather that’s always denoted by the sequence $\alpha\phi$, the first being the symbol for 1 and the second for zero. To avoid confusion, lets stick the system of base 3.
Also, we now can prolong the sequence of symbols, for as long as we want. Or in other words, congratulations to us, we’ve just built a positional number system!^{3}
Well, given that this post is already quite lengthy, showing how to actually use this system to build simple addition and multiplication procedures will be left to another post. This unfortunate, because the real power of the zero, so to speak, is how much it simplifies handling with numbers.
However, even if such were not the case, the positional system would be better than the alternatives. Why? Because, has already mentioned, it can prolonged, literally, ad infinitum. None of the other alternatives share this characteristic. Consider for example, the system of the romans. The biggest group for which they have a symbol is 1000: M. So 2000 is MM, 3000 MMM and 4000 would be MMMM, but that’s too much. So they up with the following convention: a dash over the symbol increases its value a thousandfold. So 4000 became . If you continue expanding to ever greater numbers, you will need to keep adding symbols or simplifying conventions—otherwise you just end up using the symbol for the largest number as the vertical bar in unary. With the same problems… With the zero(-based number system), you’re golden.
There is some evidence that humans favoured 10-based number systems because we have 10 fingers in total. For the sake of simplicity, lets suppose that for some mystical reason, the humans in this story favoured 3 instead. ↩
This is not necessarily a problem, but it does mean that achieving a “one-to-one” correspondence gets trickier. So for the sake of simplicity, lets try to find a scheme which offers us an unique representation for a given number. ↩
The choice of words in “a positional number system is intended: as the reader might suspect from the examples with groups of size 3 and 4, we can build a similar system with any base: 5, 6, etc. Some cases are specially important particularly in computer science: base 2 (binary system), 8 (octal) and 16 (hexadecimal). ↩
I few months ago I gave a proof of the generalised associative property. But although correct, as I re-read it, I got away with the feeling that it does not convey the meaning I wanted it to: why does it work? How does the conclusion follow from its premise? So I decided to illustrate how one would tackle the problem in the reverse direction, i.e. if one did not know the answer in advance.
So the base case of the associative law is that $a \cdot (b \cdot c) = (a \cdot b) \cdot c$. To see how this might generalise, lets see what happens when we have four terms $a, b, c, d$. With these, we can lay parenthesis in one of the following five ways:
Do all of these expressions give the same result? If we had concrete numbers, we could “do the math” (i.e. the calculations) but, given that mathematicians largely suck at crunching numbers, lets try to take a more abstract approach.
Consider expression 1.: you can easily transform it into expression 2., considering $(cd)$ as one term, and $a$ and $b$ as separate terms. The base case then immediately gives that $(ab)(cd) = a(b(cd))$. And considering $b$, $c$, and $d$ as separate terms we transform expression 2. in expression 3. Now taking $a$, $(bc)$ and $d$ as separate terms, we go from expression 3. to 4. Next, taking $a$, $b$ and $c$ as separate terms we go from 4. to 5. And finally, taking $(ab)$, $c$ and $d$ as separate terms, we go from 5. to 1. Thus all five expressions yield the same value and, albeit not a formal proof, this very strongly suggests that the same happens for five, six, etc. terms.
So how shall we proceed to turn the above argument into a formal proof? We have already seen that the particular placement of parenthesis chosen is irrelevant for the base and “base + 1” cases. What about the case of an expression with $n$ terms? Specifically, can we prove that regardless of parenthesis layout, the result is always the same? In the reasoning above, the key was to take an expression with four terms and considering some sub-expressions as terms on their own, thereby obtaining an expression with fewer terms, where the base case applied. But clearly the method of proving that every possible parenthesis layout is equivalent to the next won’t do it, because it is not scalable.
So we have to attack the problem from a different angle. What if we could prove that, irrespective of the number of terms and parenthesis layout, the result was always equal to the same sequence of terms, but with a specific parenthesis layout? OK, we can give this a try, but first we have to chose a particular layout. The one used in 1. is a bad choice, because it doesn’t apply when the number of terms is odd. The ones used in 3. and 4. are awkward to generalise, because you have to lay parenthesis at different “places” as the number of terms grows. That leaves 2. and 5., which expand by wrapping the whole expression in parenthesis and adding the new term to the left or right, respectively. Departing from my choice in my previous post on this issue, I’ll use 5, for it simplifies the recursive definition. This yields the following induction hypothesis:
Let $a_1 \cdots a_n$ denote a sequence of $n \ge 1$ terms, with an arbitrary parenthesis layout. Then $a_1 \cdots a_n = \prod_{i=1}^n a_i$. Note that this implies that we can re-arrange the parenthesis in any (valid) layout, without changing the result.
where
For $n=1, 2, 3$ this is true by definition (being trivially true for $n =1, 2$), and for $n=4$ it corresponds to form 5. above. So lets assume the hypothesis holds for a general $n$, and see what we can dig out about the $n+1$ case.
Consider $a_1 \cdots a_n \cdot a_{n+1}$. Following the reasoning of the previous post, there is an outer $\cdot$ operation, that splits that sequence into two smaller sequences. E.g. in 1, those two smaller sequences are $(ab)$ and $(cd)$. But alas, we hit an obstacle: the original sequence has size four, but the sub-sequences have size two! This is a general problem, which means we have no guarantee of being able to use the $n$ case as a building block to the $n+1$ one. This means we have to use strong induction.
Although it can proven to be equivalent to the so called simple induction, in strong induction one assumes the induction hypothesis is true for all (positive) integers less than or equal to $n$, and then proceeds to prove the $n+1$ case. The proof is now straightforward, as we can re-write the sequence as: , $k \ge 1$. But each of the sub-sequences are of size less than or equal to $n$, which means the induction hypothesis applies. Thus:
Notice that the induction hypothesis allows us to change the parenthesis layout for any sequence of up to $n$ terms. It is for this reason that we can re-write the last $n-k+1$ term sequence as . If this seems a bit foreign, consider our initial sequence, $a \cdot b \cdot c \cdot d$. Irrespective of the parenthesis layout, we can always re-arrange it in order to get: $a \cdot (b \cdot c \cdot d)$—the parenthesis layout of the last three terms being arbitrary, as usual. Now we can continue in a similar fashion, “pushing to the left” , , and so on, until we obtain:
which finally proves the proposition.
]]>So what led me to ditch Wordpress for Octopress? Well, first and foremost, the ability to write the posts in vim! Add to that the ability to fully preview them offline, and that’s a pretty sweet deal! And you don’t even have to lose all the previous content, because there are tools suited for the transition! (In the end, what I did loose were the comments, but more on that below).
It’s not all roses though: there’s a reason why Octopress is described as a blogging framework for hackers. You have to do some non trivial set up, and even the “writing routine” involves some steps the casual (or non technical) user might not expect (and be willing to put with). For example, categories for a given post have to be written down by hand, instead of selected by ticking a box. But nevertheless, it’s much much easier to customise than Wordpress (trust me, I’ve done both), and it’s written in Ruby, which I… well, let’s just say I dislike it a lot less than I do PHP. And to top it all, the “publishing” process is done on your computer, and the only thing that gets uploaded to the server are static pages. Can’t get any simpler (or securer, or faster) than that!
In what follows, the instructions are for my distro of choice, Arch Linux. Adapt as needed. At this point I should mention that there is one thing I use but won’t cover here: disqus. It is really easy to set up for Octopress, and so, if you wish to use it, a few google searches should more than suffice.
Details are below the fold.
The first think to notice is that Octopress runs only with a reduced
set of Ruby versions. So the first order of business is set up Ruby.
This basically consists of installing a small piece of software
that, for a given location, fakes the ruby infrastructure that
actually is installed in the system. Then install (clone) Octopress
into that location (here we assume ~/octopress
).
$ git clone git://github.com/sstephenson/rbenv.git ~/.rbenv
$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.zshenv
$ echo 'export PATH="$HOME/.rbenv/shims:$PATH"' >> ~/.zshenv
$ echo 'source "$HOME/.rbenv/completions/rbenv.zsh' >> ~/.zshrc
$ git clone git://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
$ git clone git://github.com/imathis/octopress.git ~/octopress
$ cd octopress
$ rbenv install 1.9.3-p0
$ rbenv local 1.9.3-p0
$ rbenv rehash
$ rbenv exec gem install bundler
$ rbenv rehash
$ rbenv exec bundle install
Unless explicitly indicated otherwise, hereinafter all the commands
assume the current directory is ~/octopress
.
Next install the default theme ($ rake install
), and check if
everything works, by powering the local server: cd
to the location
where Octopress is installed (if you’re not there already) and $
rake preview
. Then point your browser to localhost:4000
, and you
should see the (admittedly rather crude) default version of
Octopress!
Moving on, like I said, the default theme is, well, crude, so lets get something better. I choose the mathy theme, which is installed like so:
$ git clone git://github.com/stchangg/mathy.git .themes/mathy
The current version of this blog is based on a heavily modified version of that theme—eventually the changes will materialise in github…
Octopress gives you the chance of deploying either to the root of
your webserver, or to some
subfolder. What I do
a mixture of both: I deploy to the root, but the blog per se is
“located” in /blog
(the root /
shows a bit of biographical
information about yours truly here). To accomplish this, set up
everything to deploy to /
, and then inside the source
directory
create a folder named blog
, and inside that an index.html
along
these lines:
---
layout: default
---
<div class="blog-index">
{% assign index = true %}
{% for post in paginator.posts %}
{% assign content = post.content %}
<article>
{% include article.html %}
</article>
{% endfor %}
<div class="pagination">
{% if paginator.next_page %}
<a class="prev" href="{{paginator.next_page}}">← Older</a>
{% endif %}
{% if paginator.previous_page %}
<a class="next" href="{{paginator.previous_page}}">Newer →</a>
{% endif %}
</div>
</div>
Now, example.net/blog/
will display your posts. The only thing
left is to change the index.html
in the root (in my case,
~/octopress
) to what you want to show at example.net
(in my
case, the bio stuff). But, given that you have deployed to the root,
the posts’ (and pages’) URL will be immediately below the root. I.e.
if you have a post named “Foobar”, the URL will be
example.net/yyyy/mm/dd/fobar/
, yyyy-mm-dd being the post’s date
(this can be any date you want, as you can set it up while writing).
Although simple in hindsight, this one was a royal pain to get working. Basic instructions are provided here.
First, you need to change the Markdown engine used by Octopress to
kramdown
. Basically this means tweaking your _config.yml
looks
something like this (on and after the line that starts with
markdown:
):
markdown: kramdown
kramdown:
use_coderay: true
coderay:
coderay_line_numbers: table
coderay_css: class
pygments: false # default python pygments have been replaced by pygments.rb
In Gemfile
change rdiscount
to kramdown
and add the coderay
line:
gem 'kramdown', '~> 0.13.8'
gem 'coderay'
Don’t worry about the coderay
parts, those will be needed (and
explained) in the next section, about source code highlighting.
Next, add the following to source/_includes/custom/head.html
(again copy from the above folder if the file doesn’t exist):
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$', '$'] ],
displayMath: [ ['$$', '$$']],
processEscapes: true,
skipTags: ['script', 'noscript', 'style', 'textarea', 'pre', 'code']
},
menuSettings: {
zoom: "Click",
zscale: "170%",
},
messageStyle: "none",
"HTML-CSS": { preferredFont: "TeX", availableFonts: ["STIX","TeX"] }
});
</script>
<script src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML" type="text/javascript"></script>
This will make sure the code to load MathJax is included into every
page. The concrete set up means inline math is delimited with a $
,
and display math with $$
. I tried setting up the alternative \(
and \[
, but it didn’t work—I have no idea why.
Much like MathJax above, it’s (relatively) easy with hindsight… The details are given here, but the basic idea is as follows:
_config.yml
as explained above.sass/custom/_coderay.scss
. If you want the same theme
I use, populate it with the contents found
here.Now, the downside of this is that to add code snippets, you can no longer use markdown’s triple backtick syntax! Instead, you will have to add them like this (relevant kramdown documentation):
~~~
def foo():
pass
~~~
{:lang="python"}
If you think this is odd, you have my sympathies. To ameliorate
the problem, what I did was add a vim snippet, so that I just have
to type ~~~<Tab>
, and all the boilerplate automagically springs
into existence! You should consider configuring some such magic in
your editor of choice!
As an endnote, I point out that there is an alternative syntax highlighting mechanism, but it’s even weirder to use, IMHO…
By default, mathy’s fonts are all loaded from Google Fonts. Meh.
Luckily for us, we can get rid of
it!
Basically, download the font you want, create an @font-face
CSS
directive for it, and change the CSS’s to use it. The previous link
gives a really detailed explanation, so there’s no need to reproduce
it here. I’ll just list the fonts I use:
Unlike the mathy theme, this blog (no longer) has a sidebar. I used it only for the category list, but have since found a better alternative: put the categories, together with the archive in a separate page (which is added to the navigation bar), and get rid of the sidebar altogether.
To create the category/archive page, inside the source
folder,
create the archive
folder; if a folder with the same name exists
within the source/blog
folder, delete it. Inside the archive
folder, create the index.html
file, with the following contents:
---
layout: page
title: Blog Archive
comments: false
footer: false
---
<h3>By category:</h3>
<div id="blog-categories">
<ul>
{% category_list %}
</ul>
</div>
<hr/>
<h3>By date:</h3>
<div id="blog-archives">
{% for post in site.posts reverse %}
{% capture this_year %}{{ post.date | date: "%Y" }}{% endcapture %}
{% unless year == this_year %}
{% assign year = this_year %}
<h2>{{ year }}</h2>
{% endunless %}
<article>
{% include archive_post.html %}
</article>
{% endfor %}
</div>
Put the following content in
~/octopress/plugins/category_list_tag.rb
(source):
module Jekyll
class CategoryListTag < Liquid::Tag
def render(context)
html = ""
categories = context.registers[:site].categories.keys
categories.sort.each do |category|
posts_in_category = context.registers[:site].categories[category].size
category_dir = context.registers[:site].config['category_dir']
category_url = File.join(category_dir, category.gsub(/_|\P{Word}/, '-').gsub(/-{2,}/, '-').downcase)
html << "<li class='category'><a href='http://erroneousthoughts.org/#{category_url}/'>#{category} (#{posts_in_category})</a></li>\n"
end
html
end
end
end
Liquid::Template.register_tag('category_list', Jekyll::CategoryListTag)
You should now have a page that display all the categories in a list
(I should mention that I don’t use tags), and after that, lists all
the posts sorted by year/month/day, in example.net/archive
. To add
it to the navigation list, add the following line to the file
source/_includes/custom/navigation.html
(it should contain an
unnumbered list):
<li><a href="{{ root_url }}/archive">Archive</a></li>
If that file does not exist then copy it from the above folder and then change it:
$ cp source/_includes/navigation.html source/_includes/custom/
And finally, to get rid of the sidebar, you basically add sidebar:
collapse
in _config.yml
, and tweak the CSS to get rid of the
remaining thin vertical line. Details
here.
First and foremost, you should really read Octopress’ blogging basics. This is the basics of the basics.
Now, one thing that comes handy when you are writing a post, is to
be able to avoid that Octopress (re-)generates the entire blog when
your want to preview it. To do this, in ~/octopress
, run:
$ rake isolate[post-name]
Now rake generate
will only generate the given post. But writing
the post’s name like that is cumbersome, so I just issue a find
command like so, and copy/paste the relevant section:
$ find source/_posts -iname "*titleword*"
To revert the isolation, just do:
$ rake integrate
Another tip concerns drafts: if a post is not ready to publish,
add published: false
to its “header” (the first lines before the
post’s content). To do this everytime you create a new post, add
post.puts "comments: true"
to the appropriate section of the
Rakefile
(it’s easy to spot; it should be near lines 100–120).
When you’re done writing, change the header to published: true
.
Finally, something that comes very handy for long posts (rather like this one…) is the more tag, also known as fold. Real simple: just apply the 3-line changes described here, and you should be on your merry way!
A final warning: as mentioned right after the title, I’m writing months after having set Octopress up. So, while I think my notes didn’t contain any major mistakes, it is possible I got some minute (or not so minute…) detail wrong. If that’s the case, drop a comment, and I’ll try to correct it.
That being said, happy writings!
]]>The first program is one that will encrypt a given file, protecting its integrity. To properly understand this, we need some concepts. To encrypt a file means to protect its confidentiality, i.e. to modify it so it looks like gibberish. But to be useful, this transformation has to be reversible, but only to “authorised” parties. The way this problem is solved is by making the encryption and decryption algorithms depend on a small piece of information, called the secret key. You need the secret key to transform the initial information (called the plaintext) into gibberish (the technical term is ciphertext)—i.e. encrypting—and you need the same key to reverse the process—decrypting.
But, it turns out, that to protect confidentiality, encrypting the plaintext is not enough. This is very counter-intuitive, but if an attacker is allowed to modify the ciphertext, he might be able to do it in such a way that will cause predictable changes in the plaintext. For example an attacker might intercept an encrypted money transfer order, and still be able to change the amount of the transfer—even though he was unable to decrypt it. To avoid this, we must protect the ciphertext’s integrity.
Message integrity—whether the message is a ciphertext, or anything else—is protected using a Message Authentication Code, or MAC for short. The MAC algorithm also uses a secret key (1), and takes as input a message, but instead of outputting ciphertext, it outputs a tag. Both the message and the tag are sent to the recipient (who must also have the key), who then recomputes the tag, and only accepts the message if this tag matches the one he got from the sender—rejecting it otherwise.
Message integrity can be used on its own (if there is no need to protect confidentiality), but to protect confidentiality, one MUST also protect integrity: there’s not much value in receiving a secret message that could have been changed while en route… So in this latter scenario, the problem arises of how to combine the techniques of confidentiality and integrity to obtain what is called authenticated encryption. There are three alternatives:
You should always (and only) use option 3., for it is the only one that does not violate the Crypto Principle of Doom: if you when you receive a message, the first thing you do is not to check its integrity, then that will somehow lead to doom. Both in options 1. and 2. the first thing the recipient has to do is to decrypt the ciphertext—ergo DOOM. More concretely, the problem with option 1. is that it is possible for a secure MAC to leak some bits of the message in the tag (a MAC is, after all, not supposed to protect confidentiality…), and the problem with option 2. is that its security depends on the specific encryption and MAC algorithms used: for some combinations it might secure, while for others it might be insecure. Only option 3. is secure for any combination of any secure encryption algorithm and any secure MAC algorithm.
We have only two more things to cover before getting to the code, viz. cipher mode and padding. Cipher mode: within symmetric ciphers—i.e. those where there is a secret key (2)—there are two varieties: stream ciphers and block ciphers. The former are less used than the latter, and also are more tricky to use correctly, so I won’t deal with them here. Block ciphers, as the name suggest, encrypt data by processing it block by block (the size of the block depends on the concrete cipher being used). If the message is bigger than the block size, this creates two problems: first, how do you apply the cipher to the whole message? Well, you split the message in chunks the size of the block file, and use the appropriate cipher mode (more on this below). But then comes the second problem: what happens when the message size is not a multiple of the block size? You use padding. But first things first.
OK, so block ciphers process the message to be encrypted one block at a time. The naïve way to do this is to encrypt each block separately, and then concatenate all the ciphertext blocks. This is called Electronic Code Book Mode, and it should NEVER EVER BE USED. There is a valid reason for providing it, but if you’re just learning about these things, avoid it. The reason is this method doesn’t hide patterns present in the plain text, as this image from the previous link readily demonstrates:
One of the most widely used modes is called Cipher-block Chaining (CBC). Which is the one I will use here. There are others, including one with rising popularity, named Countermode (CTR). There are several versions of countermode, but they are all simpler and faster than CBC, but have a big drawback: both CTR and CBC require the user to generate some random data, usually known as an Initialisation Vector (IV) (CTR uses this IV as a counter). The problem is that with CTR, this random data must be unique: once used, it can never be used again—otherwise you get what is called a two time pad, which is a pretty nasty form of cryptographic DOOM. With CBC, re-using this is nowhere near so problematic. However, even in CBC mode, the IV must be unpredictable: the adversary must not be able to guess what the IV of a future message will be, or—you’ve guessed it—DOOM ensues.
OK, now we’re ready to code! Read below the fold to continue!
The file pycrypto_intro.py
takes a file named test.blocks
which
should already exist in the same folder as the python script, and
encrypts it to file test.blocks.enc
(asking the user for a
password). It then proceeds to decrypt this latter file to
test.blocks.enc.dec
. Thus you can compare the original and the
decryption are indeed the same. Also you can modify the ciphertext
to see that the modification is detected. Coded as it is it’s not a
very useful script, but it fulfills its purpose of illustrating how
to use PyCrypto.
The first thing to do is to take the password and use it to generate an array of bytes for the key (do note that the size of the array is the sum of the AES and the HMAC keys; in other words, two different keys are used for encryption and MAC-ing):
1 2 3 4 5 6 7 8 |
# 64-bit salt (#XXX is OS-grade randomness enough here?) salt = bytes( [random.randint(0, 0xFF) for i in range(8)] ) pwd = bytearray(getpass.getpass().encode('UTF-8')) # encoding choice is irrelevant: we only care about the bytes key = bytearray(PBKDF2(pwd, salt, 64)) # 64 byte sequence: first 32 for AES, next 32 to HMAC for i in range(len(pwd)): pwd[i] = os.urandom(1)[0] # overwrite password with random del pwd # better safe than sorry... |
Note that in the above code, the salt has to be concatenated to
ciphertext (in much the same way the MAC tag is: see
below)—something I never got around to doing in the final full
script I link to at the end of the post… Also, I should add, I use
a bytearray
for the password, so I can overwrite it when it is no
longer needed (in Python the programmer isn’t allow to do direct
actions on the memory, like deleting content). However, for
whatever reason, PyCrypto’s API requires the key to be in a bytes
object, which is immutable (so you can’t overwrite it). So my
hackish solution for this is to convert the key to bytes
when
invoking the encryption code:
1 |
encrypt_file(bytes(key), "test.blocks") |
I don’t know how effective this is as far as security goes, but it was the best I could come up with, given PyCrypto’s constraints.
Next, the encryption code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
iv = bytes( [random.randint(0, 0xFF) for i in range(AES_BLOCK_LENGTH)] ) encryptor = AES.new(key[:32], AES.MODE_CBC, iv) # SHA256 yields 32 bytes tag sig = HMAC.new(key[32:], None, SHA256) with open(in_filename, 'rb') as infile: with open(out_filename, 'wb') as outfile: outfile.write(iv) sig.update(iv) while True: chunk = infile.read(chunksize) if len(chunk) == 0: break elif len(chunk) % AES_BLOCK_LENGTH != 0: chunk = pad(chunk) encrypted_chunk = encryptor.encrypt(chunk) outfile.write(encrypted_chunk) sig.update(encrypted_chunk) outfile.write(sig.digest()) |
Pretty straightforward: generate the IV, instantiate the object that will do the actual encryption (the algorithm will be AES), instantiate the object that will calculate the MAC (we will use HMAC), write the IV, encrypt the file chunk by chunk, and update both objects as you go. Finally, append the MAC tag to the ciphertext. NB: It is not necessary to read the file in chunks, but this makes the code more generic, in that this way the program can encrypt files bigger than the computer’s physical memory. Otherwise, the whole file would be loaded into memory, which means that, for example in 32 bit machines, you’d be restricted to files with less than 4Gb.
Now for decryption, the first part is to re-calculate the MAC, and check that it matches the one we got with the ciphertext—otherwise it is rejected.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
filesize = os.path.getsize(in_filename) file_boundary = filesize - MAC_LENGTH # compute the MAC until this point sig = HMAC.new(key[32:], None, SHA256) with open(in_filename, 'rb') as infile: chunk=b'' while infile.tell() + chunksize < file_boundary: chunk = infile.read(chunksize) sig.update(chunk) # read rest of file (contains remaining ciphertext and appended MAC) aux = infile.read(filesize - infile.tell()) mac = aux[-MAC_LENGTH:] # last 32 bytes of file sig.update(aux[:-MAC_LENGTH]) # bytes from chunk except the last 32 if mac != sig.digest(): raise Oops("Decryption failed: bad MAC") |
Once we know the ciphertext’s integrity is correct, we can proceed to the decryption proper:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
with open(in_filename, 'rb') as infile: iv = infile.read(AES_BLOCK_LENGTH) decryptor = AES.new(key[:32], AES.MODE_CBC, iv) with open(out_filename, 'wb') as outfile: while True: chunk = infile.read(chunksize) if len(chunk) == 0: break outfile.write(decryptor.decrypt(chunk)) with open(out_filename, 'r+b') as outfile: # read last AES block (16 bytes) plus 32 bytes MAC outfile.seek(-(AES_BLOCK_LENGTH + MAC_LENGTH), os.SEEK_END) chunk_with_padding = outfile.read(AES_BLOCK_LENGTH) chunk = unpad(chunk_with_padding) # re-write the last block, and trim file to ignore MAC outfile.seek(-(AES_BLOCK_LENGTH + MAC_LENGTH), os.SEEK_END) outfile.write(chunk) outfile.truncate() # update the EOF indicator |
Again pretty much straightforward, except one thing: inside the
while True
loop, the entire ciphertext is decrypted, including the
padding and the MAC tag! This is to simplify the code: the loop runs
until the end of the file, and after that we read the decrypted last
AES block, unpad and write it back to the outfile (i.e. the
plaintext). We finally truncate it to get rid of the “decrypted” MAC
tag and extraneous padding bytes, and presto, we have the original
plaintext! Here’s the full source code!
A final note on randomness: although not shown in code snippets, the random generator used was not Python’s native, but PyCrypto’s cryptographically secure version of Python’s random module:
1 |
from Crypto.Random import random |
Doing this is good general advice: cryptography depends crucially on random number generation, and most random generators, even if perfectly adequate to other tasks such as simulation and what not, are not suitable for cryptography! This goes for the generators of your favourite programming language, operating system, etc. And as it turns out, even the “cryptographically secure” random number generators in widespread use today, might not be so secure after all, but I digress…
(1) - But it MUST be a different key than the one used for
encryption/decryption, otherwise, the security might depend on the
specific algorithms used. So you’re gambling your security…
(2) - The other type of ciphers are the asymmetric ciphers, also
called public key ciphers, where there is a public and a private
key. But that’s for another post.
The essence of mathematics is the construction of logically correct proofs for general theorems. A beginning student is apt to look upon a mathematical proof as a sort of magical incantation which somehow gives truth to a theorem. Nothing could be further from the intention of the person who devises the proof. A proof is worthless if it is not convincing, at least to an intelligent person who makes the effort to understand it. Generally speaking, there are two steps leading to the understanding of a mathematical proof. The first step is the mechanical checking of the proof to see that each statement follows as a logical consequence of statements which precede it. If the argument survives this test, and if the final statement is the assertion which was to be proved, then it must be admitted that the proof is valid. But to really understand the proof it is necessary to take the second, more difficult, step. One must look at the overall pattern of the argument and discover the basic idea behind it. The ideal is to see the proof through the mind of the person who originated it. Of course, this may require a high degree of mathematical talent, to say nothing of hard work, but the reward in self-satisfaction is substantial, every bit as great, perhaps, as the reward which a musician obtains from mastering a difficult piano or violin sonata.
On a (semi-) related note, understanding proofs really is at the core of gleaning mathematical beauty. That link contains a simple and nice proof—one that illustrates very well what it means to understand a proof—to the answer of the following question: in a tournament with single elimination, if you start with say, 32 players, how many games are played in total? And for any number of players?
And like so many other forms of art, the more you understand mathematics, the more beautiful it gets!
(*) - The ebook is available on slideshare, google is your friend.
]]>error: too many arguments to function ‘fuse_main_compat2’
Which is really strange, because I’m not even calling that function! So I googled until I found another hello world example, but after reading its source code, I still could not discover what was it that I was doing wrong. At this point, I should mention that the code of the examples compiled and ran perfectly. So the error had to be with my code! But where?!
So I painstakingly resorted to adding bits and blocks from the original example, until I got the darn thing to compile (which I know would happen, because the original compiled with neither error nor warnings). Eventually I discovered I was missing this line:
#define FUSE_USE_VERSION 26
But it only worked if it was at certain places! After some trial and error I discovered that that line had to be before including the fuse.h header, otherwise all hells breaks loose!
So now I had fixed my problem, and cursed the (lack of) documentation, and moved on… only to discover shortly after that that version line requirement was indeed documented… IN THE FUSE.H HEADER ITSELF!! Fusing bastards…
#ifndef _FUSE_H_ #define _FUSE_H_ /** @file * * This file defines the library interface of FUSE * * IMPORTANT: you should define FUSE_USE_VERSION before including this * header. To use the newest API define it to 26 (recommended for any * new application), to use the old API define it to 21 (default) 22 * or 25, to use the even older 1.X API define it to 11. */ #ifndef FUSE_USE_VERSION #define FUSE_USE_VERSION 21 #endif
(In all honesty I would likely have done the same, but lets gloss over such trivialities, lest we ruin the exposé…)
]]>