# On Imaginary Property

Last year, for one of my PhD classes, I ended up writing an essay1 where I criticised the notion of intellectual property. It ended up being published in Pirate Times2, which was nice, and that was—or so I thought—the end of the story. Fast-forward one year however, and some of my colleagues made the… ahem… “suggestion” that for one of the lectures of this school year’s edition of that class, I give a talk about that essay I had written.

And give the talk I did (slides). Because the students had the day filled with lectures, to avoid boring them I gave this one in a more provocative style than I might otherwise have done (and hence the name of this post—and of the talk’s unofficial title). The discussion was indeed very lively, and in fact, at times it became more heated than what I had expected. In retrospect though, I did one mistake. When arguing these things, it is usually necessary to systematically deconstruct a number of unexamined assumptions the audience naturally brings to the discussion. But because I accepted (and indeed posed) questions during the talk, that deconstruction got interrupted again again, severely derailing the course of the talk I had planned. In retrospective, I should not have allowed questions, except at the end.

Nevertheless, I am grateful for all the feedback I have received. One of most poignant remarks, was actually made by a colleague of mine, who also attended the lecture. He pointed out that I do not have all the details of how things should function if we ditch copyright in particular, or intellectual property in general (which is correct); and thus I should not outright dismiss them, at least not before being able to provide a suitable alternative. This last part however, is not correct, and here is why.

As far as I can tell, all individual rights are (or can be thought of as) restrictions on the behaviour of everyone else (other than the individual in question). Thus my right to life is a restriction on the behaviours of everyone else, prohibiting them from depriving (or attempting to deprive) me of my life. The same is true of freedom of speech—it is a restriction that prohibits behaviours from everybody else (historically this meant governments), aimed at silencing me whenever they don’t like what I am saying—freedom of religion, and so on.

Put another way, one individual’s rights are always abridgements on the liberty of others. But it falls on the proponents of a given right, to show that the corresponding abridgement of liberty is necessary. It is not those whose liberty is to be putatively abridged that need to show why it ought not to; or that have to provide “suitable alternatives”. Historically this has always been so; in fact J.S. Mill’s influential essay, On Liberty, which I quoted in the lecture, was written precisely to justify why the rights he supports should indeed be implemented. Just to mention one example, he defends the right to freedom of expression, not because the State—that in 19th Britain was the major threat to that freedom—had failed to show that it had any reason to suppress it, but rather because said freedom is vital to the full development of the individual, and that this is sufficient to justify limiting (restricting) the State’s actions in this particular regard.

Back to copyright and ilk, if they are to be construed as author’s rights, then it falls on the proponents of said rights to show why the rest of us should tolerate the corresponding abridgements of liberty—and most definitely not the other way round! Insofar as I know—and yesterday’s lecture seems to have confirmed it—that reason boils down to economics: how should authors make money, without the aid of copyright et al.? I attempted to show, on both the lecture and the essay, there’s mounting evidence that that would not be a problem at all. And thus there is no need whatsoever to abridge anything. But since we are talking about rights, I might as well address the deeper flaw afflicting that copyright-because-economics rationale.

“Author rights” such as copyright are, unlike their moral rights, established for the sole purpose of enabling (or easing) business transactions. But their effective enforcement can only be done by trampling over far more fundamental rights, two of which are the right to privacy and the right to due process (if the reader disagrees, please feel free to provide an effective counterexample enforcement mechanism; I know of none). The usual retort to this is to say that we need to find a “balance”, between the interests of authors and friends, and the rest of society that would like to maintain said fundamental rights intact. This is however, woefully misguided—and I can show so by recalling a similar, albeit far more extreme, example of the same reasoning, that took place in the Southern states of the U.S., in the early-to-mid 19th century.

Back then, the economy of those states was largely based on slave labour. When the Abolitionists began propagandising the, well, abolition of slavery, what did the defenders of the status quo (meaning slavery) replied? That «the sudden end to the slave economy would have had a profound and killing economic impact in the South where reliance on slave labor was the foundation of their economy. The cotton economy would collapse. The tobacco crop would dry in the fields. Rice would cease being profitable»3. This seems outrageous to us today, because we hold the right to freedom to be so fundamental, that it vastly outweighs any economic considerations. In other words, freedom comes first, and if that damns business, then so be it. But if, for some reason, one were to become convinced that freedom actually isn’t that fundamental, then the argument espoused by the Southerners suddenly becomes a lot more palatable.

And that is precisely the problem with the thesis of “finding balance” between authors and society at large. It seems palatable, only because most of us still do not realise how much the enforcement of those “IP rights” imperils rights that are far, far more fundamental. Maybe this is because the net and computers are still relatively new mediums; but be that as it may, those rights should not be sacrificed for the mere sake of economic convenience. And if that damns business, then so be it. In fact, even if that damns culture, so be it. Or just much cultural enjoyment do you think you will have when we are all living in a privacy-less 1984-style world?

Thankfully though, we need not face so bleak a choice. We can have both culture, and the freedom to enjoy it. To be sure, we seem to be going in a direction where cultural proliferation is increasingly less likely to yield good ancillary businesses (think selling copies of stuff). But that proliferation, that development of our collective culture, will continue, there is every indication, long after these intellectual property things have become but mere footnotes in the annals of history.

So, to sum up, I indeed do not have the complete picture of how the post-copyright-et-al world will look like; but that is no excuse for us not to ditch the bloody mess.

PS: If the reader is about to object that most of the objections I put forth above only apply to copyright, I concede as much. This is because, I suspect, as he was talking about intellectual property, my colleague whom I refer to above must have been thinking about copyright. Yet another folly that results from using the redundant and misleading notion of intellectual property… (and if you’re wondering where the adjectives come from, see footnote #1).

# (La)TeX tidbits 3: mathematics

On the previous two posts on this subject, an impromptu division of sorts came up: the first one turned out to be mostly about tweaks to articles, and the second one the same for presentations. This actually a nice scheme, and thus here’s the third one about the missing piece: mathematics.

### Aligning with ‘=’ sign

This is in most cases simple to do; the exception might be when one of the = signs, usually the first one, needs to be invisible. The below code solves that problem.

 1 2 3 4 5  \begin{align*} &\phantom{{}={}}F(x\mid\mid x' + a\mid\mid a')\\ &=F\left [ (x+a) \mid\mid (x'+a')\right ]\\ &=f(x+a) \mid\mid f'(x'+a') \end{align*} 

The result is like so:

The difference between align and align* is that the latter suppresses equation numbering.

### Math “blocks”

To get, for instance, this:

you would do this:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17  \begin{displaymath} \begin{aligned} \begin{cases} x \equiv 56 \pmod{13} \\ x \equiv 33 \pmod{23} \\ x \equiv 45 \pmod{29} \end{cases} \end{aligned} \Leftrightarrow \begin{aligned} \begin{cases} x = 56 + 13 \alpha \\ x = 33 + 23 \beta \\ x = 45 + 29 \gamma \end{cases} \end{aligned} \end{displaymath} 

To cut a long story short, the aligned environment is to be used when you want a single equation number for several lines—which typically happens when using the environment in the context of some larger construction.

Note that inside the cases environment, no &= is required for alignment—unlike what happens with align*, shown above.

### On LaTeX’s mathematical spacing

For spacing: In a “math” environment, LaTeX ignores the spaces you type and puts in the spacing that it thinks is best. LaTeX formats mathematics the way it’s done in mathematics texts. If you want different spacing, LaTeX provides the following four commands for use in math mode:

 1 2 3 4 5 6 7  \; - a thick space \: – a medium space \, – a thin space \! – a negative thin space 

## MathJax

Sometime ago I wrote about to get MathJax working locally. Now here’s another thing you might need to do in MathJax: define new math commands. No worries:

 1 2 3  
$\newcommand{\altprod}{\mathop{\widetilde{\prod}}\nolimits}$


# (La)TeX tidbits, part 2

## Presentations

I use the projector class for my presentations. Why? Glad you asked. First, these days I’m using LaTeX for so many things, that it seemed rather natural to use it also for presentations. Don’t get me wrong, it is not without its drawbacks… for example being picky about the exact layout of things, in a presentation done in LaTeX, might not be the brightest of ideas… But my layouts tend to err on the side of simplicity (more on this in a moment), and furthermore, the slides tend to include quite a bit of mathematics – which is where LaTeX can really excel. In what simplicity is concerned, I think most slides these days contain WAY too much text:

The plain absurdity of modern academic talks would be glaring if we hadn’t all pickled in it for so long. Recall the last one you attended. The speaker flashed a slide full of words on the screen and talked. Did you read and understand the slide? Did you hear every word spoken? No, you had to make a decision about what to miss.

The slides should be something to help the audience understand what you are saying, rather then aesthetically pleasing graphical compositions, which only aid the audience in focusing away of what you are saying. Because of this simplicity, the precise layout ends up not being terribly important (you have less stuff on your slide, so you have more choices on how to arrange them). Or as the UNIX world saying goes, less is more!

Having made the case for using LaTeX, the elephant in the room is why I decided to use an obscure template, instead of something like the well-known beamer class. projector’s author, Victor Shoup, actually has that one covered: «I didn’t feel like reading the beamer manual, so I just made my own». As for me, well, projector’s manual is several orders of magnitude smaller than beamer’s, so…

The basic skeleton of a slide is like so:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20  \begin{slide} \Stripe{Slide 1} \vfill { \fadeFrom{2}{85} \item Lorem ipsum dolor sit amet } \vfill { \fadeTo{1}{85} \fadeFrom{3}{85} \item Consectetur adipiscing elit } \vfill { \fadeTo{2}{85} \item Sed do eiusmod tempor incididunt } \vfill \end{slide} 

The way this works is that it generates a slide, with the (imaginative…) title “Slide 1”, and with three bullet points. But these bullet points are not displayed all at once; rather they are shown in three overlays. In the first overlay, only the first bullet is shown “normally”; the other two are shown (in this case) with an 85% transparency. For the next overlay (in practice, the next “slide”), only item two is shown “normally”, whilst the first and third are shown with the transparency. The third overlay is similar, but only the third bullet is shown “normally”, i.e. with full colour.

The reason I do this, is to on the one hand, avoid distracting the reader with too much information (usually only one of the bullets is relevant at any given time), but on the other, to allow me to use the slide as a “memory card”: while an 85% transparency is sufficient to make the text essentially not visible—and hence not distracting—to the user, it still allows me, being much closer to the projection (usually), to be able to read that text, thus being able to keep track of what I have to say next.

So in essence, the above code for the slide shows one bullet at a time, from the first to the last.

#### Showing a bullet whilst keeping the previous visible

This may seem very easy: just delete \fadeFrom{3}{85} in the second bullet. The problem is that if you do this, the program will have no idea you want three overlays. It will use just two, which means the last bullet will never be shown. You solve this by, in addition to removal of the \fadeFrom line, adding a \showAt command:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20  \begin{slide} \Stripe{Slide #1} \vfill { \fadeFrom{2}{85} \item Lorem ipsum dolor sit amet } \vfill { \fadeTo{1}{85} \item Consectetur adipiscing elit } \vfill { \fadeTo{2}{85} \showAt{3} \item Sed do eiusmod tempor incididunt } \vfill \end{slide} 

Now in the second overlay, only the second bullet will be shown; but on the third overlay, both the second and the third bullets will be shown.

#### Custom \bullet’s and \topic’s

To allow some variability in the type of bullets, I use some custom commands (that go in the preamble)…


\bulletA is an ∗, \bulletB is the regular round black-filled bullet ∙, \bulletD is ⋄, and \bulletT is ▷. Additionally, there is also a numbered bullet, which takes its number as an argument. As my presentation don’t have very long sequences, I doubt this will ever become cumbersome.

Lastly there is the \topic command, which simply uses projector’s \Banner. I use it to group bullet by topic, as can be seen in the example mentioned at the end of the post (the rename helps me remind of the command I want).

#### Continue an enumeration, itemize, etc. into next slide

I am not a fan of using those structures in presentations, but should the need arise, you can do it as illustrated here (you need to define the counter in the preamble \newcounter{tmpc}:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20  \begin{slide} \Stripe{Spill into next slide} \begin{enumerate} \item Foo \vfill \item Continue enum into next slide \vfill \setcounter{tmpc}{\theenumi} \end{enumerate} \end{slide} \begin{slide} \Stripe{Enough already!} \begin{enumerate} \setcounter{enumi}{\thetmpc} \item Bar \vfill \item Baz \end{enumerate} \vfill \end{slide} 

#### Verbatim!

projector’s manual states that, for “TeXnical reasons” (ipsis verbis), the verbatim environment cannot be used inside a regular slide. Instead one has to use the slide* environment. Here is an example:

 1 2 3 4 5 6 7 8 9 10 11 12  \begin{slide*} \Stripe{Verbathimgs} \vfill \bulletB Not verbatim text, vs.\ \verb+verbatim text+ \vfill \bulletB The \verb+verbatim+ \emph{environment} also works: \vfill \begin{verbatim} Lorem ipsur blah blah... \end{verbatim} \vfill \end{slide*} 

#### Theorems et al.

Dump the following in the preamble:

 1 2 3 4 5 6 7  % math theorem environments \newtheoremframe{theorem}{Theorem} \newtheoremframe{corollary}{Corollary} \newtheoremframe{lemma}{Lemma} \newtheoremframe{definition}{Definition} % 'def' cannot be used as environ name \theoremstyle{remark} \newtheoremframe{remark}{Remark} 

Now to show, e.g. a theorem, do:

 1 2 3 4 5 6  { \fadeTo{3}{85} \begin{theorem}[(Euler's identity)] % Note the ( and ) in the optional name $e^{i\pi}-1=0$ \end{theorem} } 

As commented in the snippet, for the theorem/etc./’s name to be enclosed in parenthesis, these must written explicitly (unlike what happens with my “regular” LaTeX setup).

#### Images / other floats

Say your image’s filename is photo.pdf (other extensions might also work; if not, imagemagick is your friend). If it is a big image, that is supposed to occupy most of the slide, you can do something like:

 1 2 3  \vfill \centering{\graphicbox{figures/photo}} % Note: file extension unneeded \vfill 

Otherwise, you can place that image in the slide like so:

 1 2  \putbox{0.65\textwidth}{0.30\textheight} {\graphicbox[scale=0.19]{photo}} % Note: file extension unneeded 

\putbox creates a box with at distance (in this case) {0.65\textwidth} and {0.30\textheight} from the right and up of the lower left corner of the slide, respectively. Adjust the scale accordingly. (For more tricks with boxes, see section 5.8 of projector’s manual).

#### Big question mark

Most of presentation end with a big smiling question mark, occupying the entire slide. It is drawn like shown below. The marvosym package (link), is needed for the \Smiley symbol—I think it is already a part of TeXLive.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20  % in preamble \usepackage{tikz} usepackage{marvosym} % for \Smiley % in document \begin{slide} \Stripe{Questions?} \vfill \begin{center} \begin{tikzpicture}[ultra thick,scale=0.5] \draw[line width=3.5,line cap=round] (1.5,0) .. controls ++(0,4) and ++(0,-2) .. (4,4) to[out=90,in=0] (2,6) to[out=180,in=90] (0,4); \end{tikzpicture} \end{center} \vfill \putbox{0.435\textwidth}{0.04\textheight}{\Huge \Smiley} \end{slide} 

#### A full example

For an example of this style of presentation, see this page, file presentation.tex and presentation.pdf. Some of the common preamble stuff (e.g. the new commands) are in a separate file, inc_presentation_preamble.tex. Enjoy!

In my writings, I virtually always refer to my hypothetical reader as “he”. The reasons for doing so were actually very well explained by none other than Richard Dawkins, in his great book “The Blind Watchmaker”. If I am ever asked about this writing habit, I wanted to have some place to point my inquirers to, and thus I reproduce the relevant passage, from that book’s preface:

I am distressed to find that some women friends (fortunately not many) treat the use of the impersonal masculine pronoun as if it showed intention to exclude them. If there were any excluding to be done (happily there isn’t) I think I would sooner exclude men, but when I once tentatively tried referring to my abstract reader as ‘she’, a feminist denounced me for patronizing condescension: I ought to say ‘he-or-she’, and ‘his-or-her’. That is easy to do if you don’t care about language, but then if you don’t care about language you don’t deserve readers of either sex. Here, I have returned to the normal conventions of English pronouns. I may refer to the ‘reader’ as ‘he’, but I no more think of my readers as specifically male than a French speaker thinks of a table as female. As a matter of fact I believe I do, more often than not, think of my readers as female, but that is my personal affair and I’d hate to think that such considerations impinged on how I use my native language.

Although English is not my native language, this seems a sensible course of action, which is why I have adopted it.

# (La)TeX tidbits, part 1

This post starts a new rubric, which is meant to write down some of the tricks that one inevitably picks up during any non-trivial interaction with LaTeX. Sadly, coderay still does not support TeX highlighting, so the output is in raw text. Hopefully it will not be too much of a hindrance.

### “Articles” with signature

For “articles” for which you need a custom signature. Must be used in one-column mode only (and at the end of the body)!

 1 2 3 4 5 6 7 8  \begin{minipage}[c]{\textwidth} \vspace{1cm} \flushright\parbox{7.5cm}{\emph{Yours sincerely}\\ \\[1.0cm] Your Truly's Full Name} \flushright\parbox{7.5cm}{} \vspace{1cm} \end{minipage} 

Done.

### Dedication page

First, either in the preamble or in an *.sty file, define the following environment:

 1 2 3 4 5 6 7 8 9 10 11 12  % custom dedication page \newenvironment{dedication}% { \clearpage % we want a new page \thispagestyle{empty} % no header and footer \vspace*{\stretch{1}} % some space at the top \itshape % the text is in italics \raggedleft % flush to the right margin }% { \par % end the paragraph \vspace{\stretch{3}} % space at bottom is three times that at the top \clearpage % finish off the page } 

I use the Lobster Two font plus italics for the dedication; in Archlinux is comes with the package texlive-fontsextra; also remember I use LuaLaTeX, so font commands for other processors will likely differ. To use it, put in the preamble:

 1  \newfontfamily\lobster{LobsterTwo} 

Then at the place where you want the dedication, put:

 1 2 3 4 5 6 7 8 9 10  % Dedication %--------------------------------------------------------- { \fontfamily{LobsterTwo}\selectfont \lobster \begin{dedication} \large I dedicate this work to my TeX book's funny character, Dr. R. MaDillo, the giant\\ on whose shoulders a good TeX book has been written. \end{dedication}} \normalfont %--------------------------------------------------------- 

Done.

NOTE: The font can also be loaded by doing \usepackage{LobsterTwo}. However this will set that font as the default roman one, for the entire document!! Beware!

### Epigraph

The Epigraph package allows for «the pithy quotations often found at the start (or end) of a chapter». The problem is that I wanted to be able to set how much of the page’s width the quotation was suppose to take, for each quotation. The vanilla package does not allow this, so based on the ideas mentioned here, I came up with the following solution. In the preamble put:


The syntax of the new command now becomes

 1  \epigraph[]{}{} 

On the optional length parameter: if omitted, the length of the epigraph will be that of either the quote or the author, whichever is largest—until the maximum of $80$% of the page’s width (the default). If not omitted, then it must be a number between $0$ and $1$, indicating the percentage of the page’s width which is to be used. Some examples:

 1 2 3 4 5  \epigraph[0.7]{The whole point of cryptography is to solve problems. (Actually, that's the whole point of computers\emd something many people tend to forget.)}{Bruce Schneier} \epigraph{IRAV IVQV IVPV}{GAIVS IVLIVS C\AE SAR} 

Done.

### Graphics

 1 2 3 4 5  \begin{figure}[ht] \centering \includegraphics[scale=1.5]{path/to/graphics/file} \caption{\label{fig:simplest}Simple graphic.} \end{figure} 

The [ht] options to the environment instruct TeX to either place the image at the place where the environment is, or, if that is not possible, then placing it at the top of a page is preferred.

### Tables, figures, etc. side by side

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23  \begin{table}[h] \centering \parbox{0.4\textwidth}{ \centering \begin{tabular}{c | c | c} + & 0 & 1 \\ \hline 0 & 0 & 1 \\ \hline 1 & 1 & 0 \\ \end{tabular} \caption{\label{tab:xor}Addition in GF2.}} \parbox{0.4\textwidth}{ \centering \begin{tabular}{c | c | c} \times & 0 & 1 \\ \hline 0 & 0 & 0 \\ \hline 1 & 0 & 1 \\ \end{tabular} \caption{\label{tab:xor}Multiplication in GF2.}} \end{table} 

Here each actual table is drawn by the tabular environment; the table environment is actually the collection of all the tables in the same “set”. The above snippet shows two tables, side by side (horizontally). This is achieved by drawing each table (i.e. each tabular environment) inside a parbox. Inside each of these there is a \centering command to have the table centred inside its parbox (instead of left-aligned, the default). Finally to have the set of two tables centred relative to the page, we use an extra \centering outside of any parbox (line $2$).

### Endnotes

Endnotes are “footnotes” that are placed at the end of the document (instead of at the end of the package). Sometimes I use the two kinds of notes… so two distinguish them I set footnotes with Arabic numbers, and endnotes with roman numerals. Preamble:

 1 2  \usepackage{endnotes} \renewcommand{\theendnote}{\Roman{endnote}} 

Endnote example:

 1  \endnote{This is an example endnote} 

Code to place at location where endnotes are to be displayed:

 1 2 3 4 5 6  \begingroup \addcontentsline{toc}{section}{Notes} \renewcommand{\enotesize}{\small} %\def\enoteheading{\section*{Notas}} % if you need to change the Note's %section title \theendnotes \endgroup 

Done.

### Margin notes

Self-explanatory feature provided by package marginnote. Preamble:

 1 2  \usepackage{marginnote} \renewcommand*{\marginfont}{\footnotesize} 

Example:

 1  \marginnote{foobar this is a margin note} 

Done.

### References, sui generis

On certain type of documents, e.g. when describing a project or elaborating a proposal, you might want to have a section with references—but these will be very general, instead of the type that you cite for a specific fact or point. What I usually do in these cases is to have a “References” section, but one that besides merely listing the references, also has some accompanying text explaining why the references are relevant for the broader document.

It begins with—when using biblatex—adding the relevant bibliography file, which contains the references to be used, as is done usually.

 1  \addbibresource{article.bib} 

Next, add a numberless “References” section, with the explanatory text and the references per se, but without the automated title:

 1 2 3 4 5  \section*{References} For the previous repertoire of songs, check the following references. \nocite{*} % can also be \nocite{key} \printbibliography[heading=none] 

If the bibliography file contains more entries than the ones you want listed, you can use \nocite with the relevant keys, to display just those references (the * causes all references in .bib file to be shown).

Done.

### Gutenberg, et al.

To reference, for instance, an online version of a book, stored in the Gutenberg site, create in the project root a file named biblatex.cfg, with the following:

 1 2 3 4 5 6  \DeclareFieldFormat{eprint:gutenberg}{% Project\space Gutenberg\space ebook\addcolon\space \ifhyperref {\href{http://www.gutenberg.org/ebooks/#1}{\nolinkurl{#1}}} {\nolinkurl{#1}}} \DeclareFieldAlias{eprint:GUTENBERG}{eprint:gutenberg} 

Now to add to the bibliography, for example, The Life and Letters of Charles Darwin, put the following in the .bib file:

 1 2 3 4 5 6 7 8 9  @book{Darwin_letters, title = "Life and Letters of Charles Darwin", author = "Charles Darwin", editor = "Francis Darwin", volume = "1", eprint = {2087}, eprinttype = {gutenberg}, year = "1887" } 

If you’re using hyperref, a link to the above url will show up automagically in the pdf. Profit! For more details, see here.

### Paragraph “separation”

When about to do a slight change of topic, I add some vertical space to mark the change, e.g.

 1 2 3 4  Old paragraph. \vspace{1em} \noindent New paragraph. 

The \noindent is there because it makes no sense to indent a paragraph if there already is vertical spacing separating it from the previous one: the indenting serves precisely to mark that separation without vertical spacing.

If the change of topic is more stark—but not enough so to justify a change in sectioning unit—then the asterisks are your friend:

 1 2 3 4 5 6  Old paragraph. \noindent\raisebox{0pt}[3em][1em]{ \makebox[\textwidth][c]{*\hspace{1em}*\hspace{1em}*}}\\ \noindent New paragraph. 

Done.

# On Shannon's perfect secrecy

Claude Shannon defined perfect secrecy to mean an encryption scheme where the ciphertext did not leak any information whatsoever about the corresponding plaintext. Or, equivalently, an encryption scheme that remains secure even in the presence of an adversary with unlimited computational power. This notion is formalised with probabilities, but to understand that, one must first understand how (and why) do probabilities get mixed-in with crypto in the first place.

The basic principle, first put forth by Kerckhoffs, is that the safety of the cryptosystem rests solely on the key. That is, you have a plaintext message, you choose a key at “random” (more on this later), and obtain a ciphertext message; and the only way of obtaining the original plaintext from the ciphertext is to decrypt it with the same key.1

First, in any “real world” scenario, the plaintexts are not all equally likely: some will be more likely than others (e.g. considering the set of all strings over the Latin alphabet, the plaintexts that correspond to (for instance) valid English will be more likely than those that are just a random sequence of characters). This means that there exists a probability distribution over the set $\mathcal{M}$ of all possible plaintexts, which very likely is not the uniform distribution. Shannon calls this the a priori probability of a plaintext, and one must assume that this plaintext probability distribution is known to the cryptanalyst (remember we assume that the only thing the cryptanalyst does not know is the key—cf. above).

Now formally, a cryptosystem consists of three algorithms, $(Gen, Enc, Dec)$. The first generates a random key, the second is the encryption algorithm that takes a plaintext and a key and outputs a ciphertext, and the third is the decryption algorithm that takes a ciphertext and a key and outputs a plaintext. A cryptosystem is consistent if for all $m$ and $k$, $Dec(Enc(m, k), k)=m$ with probability $1$. The algorithm $Gen$ also implicitly defines a probability distribution over the set of possible keys, $\mathcal{K}$, and furthermore, one can always assume that that distribution is the uniform one.2 The probability distributions over $\mathcal{M}$ and $\mathcal{K}$, together with the encryption algorithm $Enc$, specify (again implicitly) the probability distribution over the set of possible ciphertexts, $\mathcal{E}$. The ensuing discussion assumes that the encryption algorithm is fixed.

Which brings us back to perfect secrecy. Shannon defined a cryptosystem as perfectly secret if, for every plaintext, the a priori probability was equal to the a posteriori probability; which is the probability that that plaintext was the one that originated the observed ciphertext. Mathematically we have the following

Definition 1 (Perfect secrecy). An encryption scheme is perfectly secret if for all $m_i \in \mathcal{M}$ and $e_k \in \mathcal{E}$, and for all probability distributions over $\mathcal{M}$, we have $P(m_i \vert e_k) = P(m_i)$3. End.

An elementary application of Bayes’ theorem shows that the previous definition is equivalent to having perfect secrecy if and only if $P(e_k \vert m_i) = P(e_k)$ (again for all $m_i$, $e_k$ and probability distributions over $\mathcal{M}$).

This is a rigorous definition, but when I first learned the concept, then read through the facts that stem from the definition, I felt some unease, due to the fact that for all the talk about probabilities, their space was never defined. So let’s do that now. This set is composed of all the triples $(m_i, k_j, e_k)$, where $m_i \in \mathcal{M}$, $k_j \in \mathcal{K}$ and $\left( e_k = Enc(m_i, k_j) \right ) \in \mathcal{E}$. There are $\left\lvert \mathcal{M} \right\rvert \times \left\lvert \mathcal{K} \right\rvert$ such tuples. The probability of $(m_i, k_j, e_k)$ is $P(m_i) \times P(k_j)$, because when the encryption algorithm is fixed, $e_k$ is fully determined by the plaintext and the key. Note that $\sum_{m_i \in \mathcal{M}, \, k_j \in \mathcal{K}} P(m_i) \times P(k_j)=1$, because $\sum_{m_i \in \mathcal{M}} P(m_i) = 1$ and $\sum_{k_j \in \mathcal{K}} P(k_j) = 1$, and multiplying both yields the initial summation.

This may seem as a pointless display of pedantry, but its value becomes obvious when one tries to understand (and calculate) probabilities like $P(e)$, where $e$ is a fixed ciphertext. (A remark about notation: values that are assumed fixed are not subscripted.) Intuitively, one could surmise it should be something like the summation of the probabilities for all possible plaintexts, each multiplied by the probability of choosing a key that encrypts that plaintext into $e$. The formalism of the previous paragraph allows us to verify this conjecture. Indeed, to calculate $P(e)$, just select all tuples where $e_k = e$, and sum their probability. We obtain

To see how this is equivalent to our intuitive guess, consider what happens if for a given $m_i$, there are two different keys (say $k_{j_1}$ and $k_{j_2}$) that encrypt it to $e$. Then we would have:

So we conclude that each $m_i$ is multiplied by the total probability of selecting a key that encrypts it to $e$—just as conjectured.

### Conditional probabilities

Both forms of Definition 1 are based on conditional probabilities. Let’s see what insight our formalism can provide on those events.

We can re-write the second summation in Equation 1 differently, by noting that each of the terms is just the probability that both $m_i$ and $e$ occur:

It is implicit that this is done for all keys that encrypt $m_i$ into $e$. This makes sense because the plaintexts form a partition of the probability space: $\Omega = \bigcup_{m_i \in \mathcal{M}} (m_i, \cdot, \cdot)$ (which is the same as saying that the sum of the probabilities for all plaintexts is $1$). On the other hand, for a given $m$, that is encrypted into $e$ by keys $k_{j_1}$ and $k_{j_2}$,

This is because $P(m \cap e)$ is the sum of the probabilities of the tuples of the form $(m, \cdot, e)$, and in our example there are two such tuples, and summing their probabilities yields $P(m) \times P(k_{j_1}) + P(m) \times P(k_{j_2})$. Dividing by $P(m)$ we get $P(k_{j_1}) + P(k_{j_2})$. This of course holds for more than two keys. But this means this sum is also equal to $P(e|m)$, and this in turn allows us to, yet again, re-write the probability $P(e)$ like so:

As far as I can tell, there is no description of $P(m|e)$ that is similar to Equation 2, because this depends on the probability distribution of the plaintext. The best we can write for $P(m|e)$ is the following, which is not simple at all…

* * *

### Back to perfect secrecy (again)

Despite all the talk about perfectly secret ciphers, the truth is that so far all the equalities shown are valid for any symmetric encryption scheme (not just for perfectly secret ones). The next two however, are only true if the cipher has perfect secrecy. First using conditional probabilities we prove another equivalent condition to (both forms of) Definition 1.

Theorem 1. A cipher has perfect secrecy, iff for any distribution over $\mathcal{M}$, any $m_1, m_2 \in \mathcal{M}$ and any $e \in \mathcal{E}$, we have $P(e|m_1)=P(e|m_2)$. End.

Proof. ($\rightarrow$) If the cipher has perfect secrecy, then $P(e|m_1)=P(e)=P(e|m_2)$, for any$m_1$,$m_2$and$e$$. ($\leftarrow$) If $P(e|m_1)=P(e|m_2)$, for any $m_1$, $m_2$ and $e$, then P(e|m)$is just the value of$P(e|m_i)$, for an arbitrary$m_i$$, because it is always the same by hypothesis. QED. Next we prove another necessary and sufficient condition for perfect secrecy. In the case of such a cipher, $P(e|m)$ has always the same value, for all $m$ (viz. $P(e)$). And because we can always assume that the keys are generated according to the uniform distribution, this means that, for a fixed $e$, and for any $m$, the number of keys that encrypt $m$ to $e$ is always the same. The next result shows the converse is also true. Theorem 2. A cipher has perfect secrecy if and only if, having fixed a ciphertext, for any plaintext, the number of keys that encrypt it to that fixed ciphertext is the same (but note this number can vary for different ciphertexts). More formally, let $e$ be a fixed ciphertext as before, and let $K_{m \rightarrow e}$ be the set of keys that encrypt $m$ to $e$. Then a cipher has perfect secrecy iff $\left\lvert K_{m \rightarrow e} \right\rvert$ has the same value, for all $m$.4 End. Note that the analog property when having fixed a plaintext is false: there can be more keys that encrypt that plaintext to one ciphertext than to another ciphertext, but the cipher can still be perfectly secret. We’ll see an example of this shortly. Proof. ($\rightarrow$) We have argued this direction informally, based on the property that, for perfectly secret ciphers, $P(e|m)=P(e)$. But we can also use $P(m|e)=P(m)$. From (3), if the cipher is perfectly secret: $P(m)$ is constant in the numerator summation, so it can be put outside it, and cancelled with the $P(m)$ of the right hand side. We thus obtain Remember the denominator equals $P(e)$. What this means is that for a given (fixed) ciphertext $e$, and for all plaintext messages $m$, the number of keys that encrypt $m$ to $e$ must have probabilities that always sum to the same value. Given the assumption of a uniform key distribution, this is the same as saying the number of keys must always be the same. ($\leftarrow$) If for any ciphertext $e$, the number of keys that decrypt it to any plaintext is the same, then we immediately have that for any two different plaintext messages, $m_1$ and $m_2$, it must be the case that $P(e|m_1)=P(e|m_2)=P(e)$. Theorem 1 now yields the conclusion that the cipher is perfectly secure. QED. ### The question After that lengthy introduction, we now finally come to the question that actually annoyed me enough to try to visualise the probability space in the way I’ve just described. That question is the following: does perfect secrecy imply the ciphertext distribution is uniform? I could not either prove it or refute it, but it turns out the answer is no. Here’s the counterexample: we have two bits of plaintext, $(b_0, b_1)$, four bits of key material $(k_0, k_1, k_2, k_3)$, and three bits of ciphertext $(c_0, c_1, c_2)$: This encryption algorithm has perfect secrecy, because for any given ciphertext, there is the same number of keys that decrypt it to any plaintext (cf. Theorem 2). This is straightforward (if somewhat laborious) to see. Consider the ciphertext $(0, 0, 0)$, and an arbitrary plaintext $(b_0, b_1)$. What are the keys that would encrypt the said plaintext into the said ciphertext? Given that $c_2$ must be zero, we have that $k_0 = b_0$. The same reasoning yields $k_1 = b_1$, because $c_1 = 0$. Finally, given that $c_0=0$ and $b_0 \oplus k_0 = 0$, it must be the case that $k_3 \land k_2 = 0$. This yields the following three possible keys for encrypting $(b_0, b_1)$ into $(0, 0, 0)$: $(b_0, b_1, 0, 0)$, $(b_0, b_1, 0, 1)$ and $(b_0, b_1, 1, 0)$. We denote this set as $(b_0, b_1, \{(0, 0), (0, 1), (1, 0)\})$. Reasoning similarly, we can write the following table, listing the keys that encrypt an arbitrary plaintext $(b_0, b_1)$ into the ciphertext in the left column. An overline ($\overline{b}$) denotes the complement of the bit $b$. CiphertextKeys $$(0, 0, 0)$$$$(b_0, b_1, \{(0, 0), (0, 1), (1, 0)\})$$ $$(0, 0, 1)$$$$(b_0, b_1, 1, 1)$$ $$(0, 1, 0)$$$$(b_0, \overline{b_1}, \{(0, 0), (0, 1), (1, 0)\})$$ $$(0, 1, 1)$$$$(\overline{b_0}, \overline{b_1}, 1, 1)$$ $$(1, 0, 0)$$$$(b_0, b_1, \{(0, 0), (0, 1), (1, 0)\})$$ $$(1, 0, 1)$$$$(\overline{b_0}, b_1, \{(0, 0), (0, 1), (1, 0)\})$$ $$(1, 1, 0)$$$$(b_0, \overline{b_1}, 1, 1)$$ $$(1, 1, 1)$$$$(\overline{b_0}, \overline{b_1}, \{(0, 0), (0, 1), (1, 0)\})$$ Thus we can see that for any fixed ciphertext, there is the same number of keys that cause it to decrypt to any plaintext; thus the scheme has perfect secrecy. However, the ciphertext distribution is not always uniform: if both plaintext and keys are assumed uniform, then (for example) the ciphertext $(0, 0, 0)$ will be more likely to appear than $(0, 0, 1)$, because there are more keys that encrypt an arbitrary plaintext to it. In other words, although the cipher is perfectly secret for all plaintext distributions, there is at least one (viz. the uniform distribution) for which the ciphertext distribution will not be uniform. Also recall the remark made after stating Theorem 2: in this cipher for a given plaintext, there are more keys that encrypt it to some ciphertexts rather than others—indeed that is the cause of the non-uniformity of the ciphertexts—but it does not prevent perfect secrecy, as this example illustrates. ### The converse question And what about the converse? I.e. if the ciphertext is uniformly distributed, then is the cipher perfectly secret? As already mentioned, the ciphertext distribution is implicitly specified by the encryption algorithm, and key and plaintext distributions. Given that we can assume the key to be uniformly distributed, if the ciphertext is also uniformly distributed we have (notice that the first summation is just another notation for writing (1)): Remember that $K_{m_i \rightarrow e}$ is the set of keys that encrypt $m_i$ to $e$, and $P(K_{m_i \rightarrow e})$ is the summation of the probabilities of those keys. If the ciphertext is uniformly distributed regardless of the plaintext distribution, then we must have $P(e) = \frac{1}{\left\lvert \mathcal{E} \right\rvert}$ . This is only possible if, having fixed an $e$, $P(K_{m_i \rightarrow e})$ as the same value for all $m_i$—we denote that common value by $P(K_{m \rightarrow e})$ (notice the subscript $i$ is gone). The uniformity of the key distribution now means for a fixed $e$, $\left\lvert K_{m_i \rightarrow e} \right\rvert$ has the same value, for all $m_i$. Theorem 2 now yields that the cipher is perfectly secret. Thus we can now state Theorem 3. If a cipher has an uniform ciphertext distribution, regardless of the plaintext distribution, then it is a perfectly secret cipher. End. The converse is false, as the above example cipher shows. An example of an encryption scheme where the ciphertext distribution is always uniform, is the One Time Pad. Notice that in this case—uniform ciphertext distribution—as mentioned above, fixing $e$, the number of keys that encrypt $m_i$ to $e$ is the same, for all $m_i$. But in addition to that, because $P(e)$ has the same value for all $e \in \mathcal{E}$, so does $P(K_{m \rightarrow e})$. This means that for a fixed plaintext $m$, we have $P(K_{m \rightarrow e_k})$ has the same value, for all $e_k \in \mathcal{E}$. Or in words, for all plaintexts, the number of keys that encrypt any particular plaintext to any particular ciphertext is the same. 1. This assumes that given one specific plaintext and ciphertext, there is only key that encrypts the former into the latter (and vice-versa for decryption). This need not be the case (even for perfect secrecy!), as we shall see. But of course, for any secure cryptosystem, even if they do exist, it should be infeasible to calculate one such key. 2. Consider a cryptosystem $(Gen, Enc, Dec)$, in which $Gen$ is a really complicated algorithm, that outputs the key according to a very non-uniform distribution. This algorithm usually takes as input a “random tape”, which outputs symbols of the same alphabet, according to a uniform distribution. Then we can use this random tape as a new key generation algorithm $Gen'$, and incorporate the operations of $Gen$ in $Enc$ and $Dec$. Thus we obtain a derived cryptosystem $(Gen', Enc', Dec')$, with a uniformly generated key, but with the same ciphertext distribution—because the operations are still the same, they are just “in a different place”. More concretely, consider a specific tuple $(m, k, e)$, with probability $P(m) \times P(k)$. Let $k'$ be the (uniformly random) key which $Gen$ used as its random tape to produce key $k$. This means that $P(k) = P(k') \times P(Gen_{k' \rightarrow k})$, where the last term is the probability that $Gen$ produced key $k$ when its random tape produced $k'$. Then, under the new cryptosystem, what is the probability of $(m, k', e)$? Remember that the previous $Gen$ has been incorporated into $Enc'$, which means that the probability is that of $m$ being selected, times the probability of $k'$ being selected, times the probability of $Enc'$ internally transforming $k'$ into $k$. I.e. $P(m) \times P(k') \times P(Gen_{k' \rightarrow k})$; but this is just the original value. In particular this means that for any $e$ and $m_i$, the value of $P(e \vert m_i)$ does not change (cf. footnote #4). As an ending remark, note that if the only source of randomness for $Gen$ is the random tape, then $P(Gen_{k' \rightarrow k})=1$, i.e. $k'$ is always transformed in $k$. 3. To be rigorous, we would have to define random variables $M$ and $e$, and say that $P(M=m_i \vert e=e_k) = P(M=m_i)$, where $m_i \in \mathcal{M}$ and $e_k \in \mathcal{E}$. But such a level of rigour is not needed here. 4. To better illustrate what the assumption of uniformly generated keys means in this context, suppose that (for some perfectly secure cryptosystem), for a given ciphertext $e$, there exists a plaintext $m_1$, for which there are two keys that encrypt it to $e$, and a plaintext $m_2$ for which there are three such keys. Furthermore, suppose that the key distribution is such that $P(e)=P(e|m_1)=P(e|m_2)$. By the method of footnote #2, if we now produce an equivalent cryptosystem with uniform keys, the ciphertext distribution does not change, so neither do $P(e|m_1)$ or $P(e|m_2)$; but they can no longer be equal, because as there are more keys for $m_2$ than $m_1$, we must have $% $. This shows the original cipher could not have been perfectly secure. # Photos dot Google When I decided that some of the photos I took were good enough to publish online, I first tried using Google Plus as a photo album, because I already had an account. It worked, but it was something that can fairly be described as the poor man’s photo album. So wanting something a little better, I tried flickr. It was good, and I stuck with it—until I logged in from abroad, and got locked out because I could not answer a phone challenge, considering how I had not given them my phone number (nor having any intention to do so). So I was back at having my photos only in my hard drive, until I heard of Google Photo. It’s free, unlimited, and it totally lacks privacy (from Google), but for the photos I intend to store there, it’s more than enough. What I had stored in flickr encompasses roughly four categories; here are the corresponding albums @ google: Now one of the problems is that (unlike flickr) I can’t list all of the albums (or all of the photos, for that matter). But I can route around that problem. As Prof. Fred Brooks pointed out, one must «[p]resent to inform, not to impress;», because «if you inform, you will impress.». So while I cannot dump all of the photos and provide a link for you to sift through, I can put the relevant new ones in (public) albums, and write a blog post embedding them, thus making them part of the story. And besides, that way I get to debut a new category! ;-) # Debian ex Arch(Linux) So I decided to use some free un-allocated space in my laptop’s hard drive to install GNU/Linux Debian. Doing this has been on my mind for some time now, but the final straw that pushed me to do it was the fact that I need to use Skype (ahem…), and given that it is a piece of crap proprietary software, now owned by an NSA friendly corporation, I decided I might as just install Debian, and dump Skype therein, where it can only cause a minor amount of damage. (Of course I only noticed that for Linux only 32 bit binaries are provided after installing 64 bit Debian…—but alas I got lucky, as that did not bring a lot of extra trouble—more on that later.) OK, so having ArchLinux already pre-installed in my laptop, I decided that the best thing to do is install Debian from Arch. What I describe next are the changes to that procedure that I have to do to get it to work. So first, this method of installation requires the debootstrap tool; for Archers, you just have to get it from the AUR. Next, after creating the chroot environment and chroot-ing into it, the PATH environment variable will not be set—which means that to run commands at the prompt, you’ll have to invoke them with their full path—not very practical. The solution is to set PATH to the value of ENV_SUPATH in the /etc/login.defs file.  1  # export PATH=  Next, when setting the timezone, there exists nothing in the /etc/rcS file that is used to indicate «whether the system will interpret the hardware clock as being set to UTC or local time». But this did not seemed to cause trouble so, moving on… Next, I stumbled across one weird error when running aptitude update. It complained that some signatures could not be verified, but it turned out that it was caused by a network outage I had failed to notice… Next on the menu is dealing with the bootloader. I use GRUB, so it was just a matter of adding an entry for Debian. The partition where Deb is installed is /dev/sda4, so in file /etc/grub/40_custom I appended the following:  1 2 3  set root(hd0,4) linux /boot/path/to/vmlinuz root=/dev/sda4 initrd /boot/path/inird/image  After that you have to run # grub-config -o /boot/grub/grub.cfg to generate the GRUB configuration with the new entry. In the snippet above, the reason you have to specify the root location twice is that in more advanced setups, the values could be different: in the first line we tell GRUB where to find the linux image (vmlinuz), whilst in the second one we tell the linux kernel where the root filesystem is located. For example if we had set up a separate boot partition, those values would differ. In the second line you can also add kernel options; the ones I used were taken from here. A quick side note about initrd: in my first attempt, the Debian system I ended up with was not bootable. I thought at first that this might be because the initrd was missing a module for ext4, the filesystem I choose to use. So I added the module, like so:  1 2  # echo ext4 >> /etc/initramfs-tools/modules # update-initramfs -v -u -k uname -r  Next, we have to deal with users and passwords. The first step is to set root’s password: just run passwd. Then you have to add an everyday, non-root user: run adduser. Next you have to install and set up sudo, and KDE.  1 2  # aptitude install sudo # aptitude install kde-standard  About KDE, when selecting the keyboard layout, two of the choices were “Português” and “Português nativo”. DO NOT CHOOSE THE LATTER! It completely screwed up my keyboard, inside KDE, to the point I was not able to login (because I could not type the password). Always choose “Português”! Anyway, that ordeal forced me to reconfigure the keyboard, which is done like so:  1  # dpkg-reconfigure keyboard-configuration  For testing, I also required the startx command (to avoid having to reboot the machine to see if my latest tweak solved the problem…). It is in the xorg package. Finally, although I only noticed this after installing Skype, sound was not working in Debian. To fix this, do in root, and reboot:  1 2  # aptitude install alsa # alsactl init  Actually, this got sound working on Debian generally—but not on Skype. More on that below. ### Skype The instructions for amd64 in Debian’s wiki are sufficient:  1 2 3 4 5  # dpkg --add-architecture i386 # apt-get update # wget -O skype-install.deb http://www.skype.com/go/getskype-linux-deb # dpkg -i skype-install.deb # apt-get -f install  The first line enables Multiverse (remember Skype is 32bit only, whilst we are now in a 64bit Debian…), and the last fixes missing dependencies. Beware to check any packages that last command might want to remove, for some of those may be potentially still necessary! Now, as mentioned before, there was no audio in Skype, and it turns out the fine wiki has the answer for this as well: Skype now requires pulseaudio to be installed. The Debian installer already installed the libraries for this; I was only missing the server, so:  1  # aptitude install pulseaudio  Notice that this will start the pulseaudio server, and set it up for starting when logging in. I end this post by noting that if you use an external webcam and/or microphone (as you should), these have to be selected in Skype after been plugged in. That’s it! Enjoy your conversations and, while you’re at it, don’t forget to thank the patriotic men and women who have nothing better to do tirelessly monitor everything we do and say, to stop The Bad Guys™ from triumphing! Have fun! # GnuPG's web of trust When reading a clarification about the way GNU Archlinux uses GnuPG, I realised that I had never quite grasped the way a web of trust actually works. I mean yeah I understood why it is needed, and what problem it attempts to solve, but all that talk about trust and validity always confused the heck out of me… I could still use GnuPG even without actually understanding it, so I usually did not bother… until today. This is my attempt to sum up my conjectures about how trust and validity actually work—and if they are correct, then I have to say the manual really sucks (even if GnuPG doesn’t! Quite the opposite in fact)1. So first we have to distill meanings: trust means trust in a key’s owner: i.e. how carefully will he check other people’s keys before signing them. Validity is a measure of the trust you have that a certain key does belong to its purported owner. Trust has the following four levels: • Unknown: Nothing is known about the owner’s judgment in key signing. Keys on your public keyring that you do not own initially have this trust level. • None: The owner is known to improperly sign other keys. • Marginal: The owner understands the implications of key signing and properly validates keys before signing them. • Full: The owner has an excellent understanding of key signing, and his signature on a key would be as good as your own. Now, as far as I can understand it, the same four levels apply to validity. So you can have a key that has full validity, marginal validity, or unknown validity—but I have no idea of what a validity of none means. The other important bit you need to realise is that trust is set by the user, whilst validity is automatically determined by GnuPG. The rules for determining it seem to be as follows: • A key that you have signed with your own key is fully valid. • A key that is signed by a fully trusted key is fully valid. • A key that is signed by (at least) three marginally trusted keys is marginally valid. • For the scenarios where you have not signed the key which validity is to be determined—i.e. the previous two—there has to exist a path from that key to your own key with length at most five. This last step means that there has to exist a sequence of keys like so: you key signs some key A, that in turn signs other key B, … until some key that signs the target key—where the path contains six or less keys (including the target and your own)—i.e. the path length is five or less. This seems to explain the results in the example scenario outlined in the above linked section of GnuPG’s user manual. I hope it helps… 1. For some reason I was overly enthusiastic when I wrote that… Yes GnuPG sucks, although by the looks of it the manual manages to suck even more… At any rate, as Mathew Green pointed out, «a PGP critic is just a PGP user who’s actually used the software for a while», so… # GPG, the saga continues For reasons that are total and utterly beyond me, my private keys vanished from my GPG keychain. Remember that I use GPG with one master keypair, and two subordinate keypairs (one for encryption, and other for signatures), and I keep the master keypair’s private key (which is needed only on seldom occasions) on an offline location. And yet, a few days I noticed that I could no longer either encrypt or sign email messages in mutt. I dug around a bit, and much to my astonishment, discovered that gpg -K produced not output at all! With (a lot of) hindsight, this was probably caused by two factors: one, the change of format for the storage of secret keys in GPG 2.1, and the fact that the ~/.gnupg/private-keys-v1.d directory, where secret keys are now kept, had no executable permissions, which means its contents were inaccessible to every user or application! Now how did that came to happen, I haven’t got the foggiest. But by the time I realised this as a likely cause for my predicament, I was already well on my way to solving the problem through the hard approach—which is what I now describe, should I ever come to need it again… OK, so remember that my “everyday” ~/.gnupg does not contain my private master key, so I keep a backup of my “full keypair” in, lets say, /media/Secure/dotgnupg. What I did was basically export from my full keypair those parts that had gone missing in my everyday ~/.gnupg, and import then therein. So, to the code, that talking (or in this case writing…) is cheap! Back up your stuff, and generate a new dummy key (to initialise the new ~/.gnupg), which we’ll get rid of at the end.  1 2  cp -r ~/.gnupg ~/gpg.old gpg --gen-key # dummy values  Then, export the missing subkeys, and re-import then in the everyday .~/.gnupg. Note that this will also export any expired (and, I assume, revoked) subkeys, thus allowing you decrypt and verify signatures on older messages.  1 2  gpg --homedir /media/Secure/dotgnupg --export-secret-subkeys MYKEYID > my.subkeys gpg --import my.subkeys  If you now list your secret keys, the only thing that should be missing (i.e. with an # in front) is the master private key.  1 2 3 4 5 6 7 8 9 10 11 12 13 14  $ gpg --list-secret-keys --list-options show-unusable-subkeys /home/vrael/.gnupg/pubring.kbx ------------------------------ sec rsa2048/81FDCED3 2014-12-15 uid [ultimate] bilbo ssb rsa2048/F5A10CA6 2014-12-15 sec# rsa4096/B58761E3 2013-02-09 uid [ unknown] Óscar Pereira (Check https://blog.erroneousthoughts.org/pgp-keys/ for up-to-date key status info) uid [ unknown] Óscar Pereira ssb rsa2048/BAB2746E 2013-02-09 [expired: 2014-02-09] ssb elg2048/A79BBD4C 2013-02-09 [expired: 2014-02-09] ssb rsa4096/2D196E9F 2014-02-23 [expires: 2019-02-22] ssb rsa4096/BA911BB2 2014-02-23 [expires: 2019-02-22] 

Now we have but to delete the dummy keypair (and the backup of ~/.gnupg)1:

 1 2  $rm -rf ~/gpg.old$ gpg --delete-key 81FDCED3 

Notice that having been just freshly imported, your keys (or more precisely your uids) have unknown validity—which is of course nonsense. To set them to ultimate do (and repeat it for uid 2):

 1  \$ gpg --edit-key B58761E3 / uid 1 / trust / 5 /save / quit 

And we are done. Two endnotes:

1. Before unmounting /media/Secure/dotgnupg, you need to killall gpg-agent, otherwise you get an error message saying that the device is busy… (also note you don’t have to restart the agent afterwords, because now it is started by gpg whenever needed).

2. If you have a lot of signatures in your key, Kgpg has a nice feature: which is to automatically import all the keys corresponding to those signatures.

1. I won’t deal with it here, but if you have a custom gpg-agent.conf file, you might also want to copy it from the backup before deletion.