Saturday, April 12, 2014

The Four Choices of the Windows XPocalypse

It's April, 2014, and Windows XP is dead.  Although XP has its detractors, it has served us very well for a dozen years.  It's unfortunate that Windows Vista was, or was perceived to be, such a dud.  It was possible to upgrade in place from XP to Vista, and more people would have done it if Vista hadn't gotten such a bad rap.  It did, and they didn't.

Now what?

There are four possibilities:
I've fully tried the last three three of them, and am messing around with the "Keep on with XP" option.  I'll write a little bit about my experience with each one, but first a recommendation: Buy a new computer and get a new operating system.

This article tells what to do, but not so much about how to do it.  None of this stuff is hard, but you might want to have a techie friend work with you through whichever option you choose.

One other thing: to accomplish either of the latter two possibilities, you will need an external disk drive, which will cost you $90 to $150.  You really need one of these anyway so you can back up your data.

Keep on With Windows XP

I can't really recommend this one.  Eventually, some bad guy will find a serious flaw in Windows XP and Microsoft won't fix it.  Do you care?  Yes, you do!  Operating system flaws put your data at risk.  Irreplaceable photographs could be deleted or corrupted.  If your web browser knows your banking password, or something equally important, the bad guys could get it.  They could send spam through your computer or mount attacks on others from your computer.

With that said, Windows XP did not turn into a pumpkin on April 8, and no hard drives exploded.  If you're a little careful, you could get several more years out of Windows XP.  How do you be careful?

Run Windows Update:  Microsoft released security patches for WinXP on April 8.  Be sure you've got them by running Windows update. If you had automatic updates turned on, you should already have the latest updates, but it won't hurt to check.  If you had automatic updates off, keep running the update process until it tells you there are no updates.  Do this now, in case Microsoft removes the XP updates from their servers.

Dump Internet Explorer:  The latest versions of Internet Explorer will not run on XP; we're stuck with IE 8, which is not being updated any more, and the web browser is the entry point for a lot of malicious software.  The most current versions of Firefox and Chrome work on XP.  Get one or both.  Do not use IE 8 for anything.  At all.

Don't be "Administrator:"  Many of us, myself included, were in the habit of using a login account with Administrator privileges. The trouble with that is, if malicious software finds its way into your computer, it will have administrative privileges, which means it can do great damage.  Make a new account, like "Bob Brown Adm" and give it administrative privileges.  Change your regular account to have limited privileges, and use that one unless you really need to administer something.

Disable Java and Flash in the Browser:  These are major entrances for malicious software, and if they're enabled, they will run automatically if you reach an infected site.  By the time you know there's a problem, it will be too late.

Run Anti Virus Software:  Although your operating system won't be updated when new security flaws are found, your anti-virus program may be able to defend you from some of them.  If you are already running Microsoft Security Essentials, Microsoft has promised updates through July, 2015.  If you don't already have an anti-virus product and need something free, try Avast! 2014.  My recommendations for paid products are those from Kaspersky or F-Secure.  Don't try to run more than one A-V product on the same computer.  They don't work or play well together.

Keep your other Applications Up to Date:  The operating system is not the only thing bad guys can attack.  If you are running other applications, keep them up to date.  Software makers may keep publishing updates, especially security updates, for their Windows XP applications for some time after Microsoft support has ended.  If there are updates, get them, regularly!  Secuinia PSI will help you find out what applications need updating and help you get the updates.

Have Good Backups:  You should be doing this anyway.  The most valuable part of your computer is generally your data.  You'll need an external drive and backup software, or one of the cloud backup services.  I like the external drive approach because it's under my control.  Power off that backup drive when you're not actually making backups, or malicious software could infect it, too. 

Stay Away from Public Networks:  If you are at home or at work, you're almost certainly connected to the Internet through a network address translation (NAT) device.  It's not really a firewall, but it does protect your computer from unsolicited network packets.  A public network could assign a registered address to your computer, which would expose it to the Internet with no protection at all.  It's not likely, but it could happen.  You have to worry about the network operator and whoever else may be on the same network, too.

Buy a New Computer, Get a New OS

This is the path of least resistance, although not the path of least money.  It's not as expensive as you might fear, though.  Buying Windows 8.1 will cost you over $100 unless you are eligible for an education discount, and you can buy a new computer with Windows 8.1 pre-installed for less than $400.  (That's a pretty minimalist computer, but you can get one that's highly capable and still be in the three-figure price range.)

You can save your data, most of it automagically, but you will have to re-install your apps.

Transfer Your Data:  With both computers connected to the same network, install the (free from Microsoft) PC Mover Express software on both computers.  If possible, use a wired network; this may take forever if you try to do it wirelessly.  Laplink has a special Ethernet cable for $10 though if you don't have a wired network available.  (By the time you add shipping, it's $22, though.)  Order it when you order the new computer, and you'll have it when you need it.  (You might be able to use an Ethernet crossover cable if you have one around.  I haven't tried this.)

Follow the directions and PC Mover Express will migrate all your data, including most application settings, to your new PC.  Plan to do this over night as it could take many hours.  When I did this, it found everything except my Lotus Notes files and the profile for the Firefox web browser.  The free MozBackup program will save and restore your Firefox profile and those of other MOzilla products, like Thunderbird.

Buy and Install Start8:  The Start8 program ($5.00) restores the start button to Windows 8.1.  Microsoft has promised to put the start button back in a later release, so you may not need Start8.  Or, you may be happy with Microsoft's "start screen" and not need Start8.

Reinstall Your Applications: You will need to reinstall you applications from the original CDs or DVDs.  If you've downloaded free software for XP, you 'll have to download it again.  And, some old software may not work with Windows 8.1.

Put the Old PC in a Closet:  Once you have everything like you want it, put the old PC in a closet for six months or so.  If you find there's something you need from it, you can haul it out, fire it up, and copy what you need to a flash drive.  If you don't discover that you need anything in that six months, you probably got everything when you copied the data.

Finally, Nuke the Old Disk:  When you are confident that you no longer need anything from the old PC, destroy any data that may be on the disk.  This is a critical step, especially if you've saved passwords in your web browser, stored credit card numbers, or otherwise put sensitive data on there.  Darik's Boot and Nuke is a free program for secure erasure of disks.  Download the image file, burn it to a CD or DVD, and boot the old computer from the CD.  Be careful with this!  It will completely erase the disk(s) in any computer from which you boot it.

Once you've done the boot-and-nuke operation, there will be nothing on the disk of the old PC, including the operating system.  You can now recycle it, donate it, or (try to) sell it.  Do not have high hopes for selling it, though.  It might bring enough to buy a lunch.

Or, you could install Linux on the old PC and keep it as a spare or donate it to a church or school.  Installing Linuz will effectively wipe any previous data.  If you want to be especially cautious, run boot-and-nuke first.

Back Up Everything and Upgrade the Operating System

You want to keep your own hardware, but run a newer operating system.  You first have to find out whether your hardware will support Windows 8.1 because you can't buy Windows 7 any more.  If you are unsure whether your system meets the specifications, you can download the "Upgrade Assistant" from that link and it'll tell you.  You will need to know whether your computer has an x86 or x64 CPU to know which flavor of Windows 8.1 to buy.  Do note that a copy of Windows 8.1 will cost you $100 or more unless you can get an education discount.  You will want to think about buying a new PC or dumping Windows in favor of Linux.

You will be able to back up and transfer your data files, but you'll have to reinstall your applications, and some of them may not work under Windows 8.1

OK... you've decided to go ahead...

Back Up Everything: You are going to blow your installation of Windows XP completely away when you install Windows 8.1.  If you have anything at all on your XP computer that you want to keep, copy those files to an external disk before you start to install Windows 8.1; otherwise, they'll be gone forever!  Remember, this applies to pictures, documents, movies, etc.  You cannot use your Windows XP applications under Windows 8.1.

Install the Operating System:  Install Windows 8.1 from the DVD you got when you bought the operating system.  When you're done, you will have a bootable computer with nothing on it except the OS and a few applications that came with it.

Buy and Install Start8:  The Start8 program ($5.00) restores the start button to Windows 8.1.  Microsoft has promised to put the start button back in a later release, so you may not need Start8.  Or, you may be happy with Microsoft's "start screen" and not need Start8.

Reinstall Your Applications: You will need to reinstall you applications from the original CDs or DVDs.  If you've downloaded free software for XP, you 'll have to download it again.  And, some old software may not work with Windows 8.1.

Restore Your Data Files:  Attach the external disk to the computer and copy the data files from your backup to their new home.  The structure of Windows 8.1 is different from that of XP.  You may want to explore around a bit before you start copying the files.

You will want to hang on to that external disk for a while before you write over it in case you missed getting something off of it.  (If you missed putting something on it, it's gone forever!)

Dump Windows and Use Linux

If your computer is not up to running Windows 8.1 and you mostly use it for surfing the web and using web-enabled email, you can run Linux and be happy.  You can even do a little word processing or maybe watch a DVD using Linux.  Your WinXP "apps" will be gone, though, along with WinXP.

Don't listen to people who tell you that Linux is only for techies.  You will find modern Linux distributions very similar to what you're used to.

Back Up Everything: You are going to blow your installation of Windows XP completely away when you do this.  If you have anything at all on your XP computer that you want to keep, copy those files to an external disk before you start to install Linux; otherwise, they'll be gone forever!  Remember, this applies to pictures, documents, movies, etc.  You cannot use your Windows XP applications under Linux.

Get Lubuntu Linux:  Lubuntu is a lightweight edition of the Ubuntu Linux distribution.  Download it using your Windows XP machine and make a bootable flash drive.  You will need to know whether you have an x86 processor or an x64 processor.  Once you have that bootable flash drive, you're done with XP!  Boot from the flash drive (you may have to play with the BIOS boot options to do that) and you will install Lubuntu Linux.

You will do the following steps after you are running Linux.  There is an enormous amount of software for Linux.  Some is for generic Linux distributions and some is specific to a particular distribution.  If you can't find a package specifically for Lubuntu, use the Ubuntu version or the generic version.

Install a Browser:  Lubuntu comes with Firefox pre-installed.  If you prefer, you can install Chrome.  When you have Lubuntu running, use Firefox to do a search for "Chrome Linux" and you'll be good to go.  (I didn't put a link in here because it won't do you any good until you're already running Lubuntu.)

Install Libre Office:  Libre Office is a cross-platform office suite with many of the features of Microsoft Office.  The best feature is that it's free!

Install VLC:  The VLC media player is also free and works with Linux.  If you have a suitable sound card, you can use VLC to listen to music.  You may even be able to watch DVDs, depending on your hardware.

Saturday, May 4, 2013

A Little About Encryption

My people have no tradition of proofreading.  —Ken White

This is the third of three posts on encrypting email.  The others are It's Time to Encrypt Your Email and Using Encrypted Email

About Encryption

Encryption mathematically scrambles the bits of your email messages or other documents so that the content is impossible to read without reversing the encryption process.  The encrypting process combines a long and random collection of bits called the key with the message to to produce an encrypted message, called the cipher text.  This is analogous to putting the message in an envelope, except that the envelope cannot be opened without the key.  The cipher text can be safely sent to the recipient electronically; even if the message is intercepted, the adversary will not be able to read it. Decrypting the message involves reversing the process using the same key, as shown in the diagram.  A system of encryption that uses the same key for both encrypting and decrypting is called secret key or or shared key or symmetric key encryption.

You've spotted the problem!  Sender and recipient must each have a copy of the key, so you have to figure out a way to get the key to your recipients securely and hope they keep it secure.  You also have to have a separate key for each person with whom you want to correspond; otherwise, all will be able to decrypt everyone else's messages.  Symmetric key encryption has important uses as we see below, but for correspondence, it doesn't scale well when used by itself.

Public Key Cryptography

In the 1970s, three groups of researchers independently invented a mechanism that uses two different keys with the same message, one to encrypt and one to decrypt.  The key usually used for encrypting is called the public key, and the key usually used for decrypting is called the private key.  Here's what's important:  a message encrypted with one's public key can only be decrypted using the corresponding private key.  You can give the public key to anyone, and they will not be able to decrypt messages that others may have encrypted with the same public key.

You can give your public key to everyone with whom you correspond.  Only you will be able to decrypt the messages they encrypt for you.  In fact, there are public key servers that will allow you to post your own public key and to look up the public keys of others.  One such key server is  You will probably want to upload your public key to that one or another.  You could try looking up the public keys of others right now by clicking on the link.

Hybrid Cryptography

There's a problem with public key cryptography: it's a lot of computational work.  Encryption using public key cryptography can take as much as 10,000 times longer than encrypting the same message using symmetric key cryptography.  Encrypting a message could take from several minutes to hours.  We probably don't want to wait even a few minutes for the encryption process.

Symmetric (shared) key encryption is much faster, with times in seconds for even very long messages.  But, as we saw earlier, the problem is how to get a copy of that shared key securely to the recipient.  The solution is to use public key cryptography to solve the key exchange problem.  The sender generates a random number that's used to create a key for symmetric key encryption, using something like the Advanced Encryption Standard (AES.)  So, encryption can be completed very quickly.  The symmetric key is called a session key when used this way because each key is used for only one message or communication session.  The session key itself is encrypted using public key cryptography and the recipient's public key.  Because the session key is short – perhaps 256 bits (32 bytes) – the time to encrypt it is minimal.  The encrypted message and encrypted session key are packaged together and transmitted to the recipient.  This two-step process is called hybrid cryptography, and is almost always the way public key cryptography is employed to secure messages from eavesdropping.

The recipient reverses the process.  The session key is first decrypted using the recipient's private key.  Then, the session key is used to decrypt the message.  Besides speeding up the process, use of session keys and hybrid cryptography actually improves the security of messages because it deprives an adversary of the chance to collect many messages encrypted with the same key.  That's important for two reasons: having a large collection of text encrypted with a single key may make the cryptanalyst's job easier, and, if the key for one message is cracked, all the messages are revealed.  (Of course, if the cryptanalyst can recover the recipient's private key, the public key no longer offers security.  But, if the cryptanalyst cracks one session key, the other messages remain secure because they were encrypted with different session keys.)

Idea: A Digital Signature

The two keys of a public key crypto key pair are cryptographic inverses of one another.  A message encrypted with one key of a pair can only be decrypted with the other key of the same pair.  In the normal course of things, Alice would encrypt a message using Bob's public key, and Bob would decrypt it with his private key.  That would keep the message confidential while it travels over an unsecure channel.

If Alice encrypts a message with her private key, Bob or anyone else could decrypt it using Alice's public key.  There's no confidentiality because the message can be decrypted with Alice's public key, and that's, well, public.  However, if we believe that Alice has guarded her private key carefully, only she could have encrypted the message. So, we can say that Alice has digitally signed a message that was encrypted with her private key.

Encrypting a message with a private key (to sign it) has the same problem as encrypting with a public key to secure it, namely that it would take a very long time.  We need a way to characterize a specific message that is shorter than the message itself.

Cryptographic Hashes

In the early days of computing, it was common to add up a series of numbers before entering data into a computer, then add them up again with the computer.  If one got the same total, that was a good indication that the data entry process was free of errors.  Sometimes the numbers were things like birth dates, where a total did not have any meaning other than as a check for consistency.  Such totals were called "hash totals."

The characters that comprise a computer message are just numbers.  In theory, we could add them up to get a hash total that could serve as a consistency check on the message. In practice, it's a little more complicated. 

A cryptographic hash function has three special properties.  First, even a tiny change in a message, like adding a zero to make $100 into $1,000, must change the computed hash code.  Second, it should be impossible, given a message, to create another message that produces the same hash code.  There exist hash codes that are thought to meet both criteria.  We can think of the computed hash code as a kind of fingerprint for a message.  Different messages always have different fingerprints, even if the difference in the messages is very small.  A hash code computed over a message is called a message digest.

The third property is that one cannot reconstruct the message given only the message digest.That's important because the digital signature exposes the message digest to an adversary.

The Digital Signature Improved

Given that we can compute a cryptographic hash, we can improve upon our idea of a digital signature.  Instead of encrypting the entire message with her private key to sign it, Alice can compute a cryptographic hash over the message and encrypt the hash code only using her private key.  Since the hash code is short – perhaps 256 bits – it, like a session key, can be encrypted quickly even with public key cryptography.  The encrypted hash code is sent along with the message to serve as the digital signature.

Such a digital signature not only authenticates the sender, it protects the message from tampering while in transit.  Here's why: Anyone can decrypt the digital signature using Alice's public key, but only Alice could have encrypted it.  Bob can verify the message by computing the hash code anew, then comparing it with the decrypted hash code sent with the message.  If they're equal, we can be sure the message actually came from Alice.  We can also be sure the message hasn't been altered.  If Evil Eve had altered the message, the hash code Bob computed would  be different from the one Alice computed.  The computed hash code wouldn't match the decrypted hash code, and the digital signature validation would fail.  (Eve can't replace the hash code after altering the message because doing so requires Alice's private key, which only Alice has.)

Putting the Pieces Together

We can encrypt a message using hybrid cryptography, and we can authenticate it using a digital signature.  If we put those pieces together, we can package a message that only the intended recipient can decrypt, and for which the authenticity of the sender is assured.

The diagram below shows what happens when Alice's encryption software prepares a digitally-signed and encrypted message for transmission to Bill.  Plain text (unencrypted) information is represented by green boxes, encrypted information by red boxes, and keys by orange boxes.  The package of information that is transmitted to Bill is surrounded by a blue box.  The digital signature is "sort of green" because, although it is encrypted, anyone can decrypt it using Alice's public key.The circle-plus symbol indicates encryption.

Alice's encryption program uses a cryptographic hash algorithm to compute a message digest of the plaintext message, then encrypts the message digest using Alice's private key to form the digital signature.  A random number generator is used to produce a session key.  The session key will be used only once, for this message.

The plaintext message is encrypted using the session key and a symmetric key encryption algorithm such as AES.  The session key itself is encrypted with Bill's public key.

There are three encryption operations.  Symmetric key encryption is used for the "main" message because it is comparatively fast.  Public key encryption is used to produce the encrypted session key and the digital signature.  This works because, although public key encryption is slow, both the message digest and the session key are small, perhaps 32 bytes each.  The "package" sent to Bill includes the three main components shown in the diagram and some additional information.  It identifies Alice as the sender, Bill as the recipient, and names the encryption algorithms used.  It is possible to include more than one encrypted session key.  If Alice wanted to send the same message to both Bill and Charlie, the package would include a copy of the session key encrypted with Bill's public key and another copy encrypted with Charlie's public key.

Bill Receives the Message

Bill gets the packaged message over the Internet by email, or perhaps in some other way. The session key was encrypted by Alice using Bill's public key.  Using Bill's private key, Bill's crypto program decrypts the session key.  The session key is used to decrypt the actual contents of the message.  Since only Bill has a copy of his private key, only Bill can decrypt the session key, and hence decrypt the message.  The message is now available in plaintext.  The next step is to check that it really came from Alice and is not a forgery.

Alice digitally signed the message by encrypting a message digest using her private key.  We know two things about that.  Since the digest was encrypted with Alice's private key, only Alice's public key can decrypt it.  Since only Alice has a copy of her private key, only Alice could have encrypted it.

Bill uses Alice's public key to decrypt the digest that is the digital signature.  Bill also computes a new message digest from the plaintext message.  The decrypted digest and the computed digest are compared.  If they are equal, Bill has confidence that the message actually came from Alice, and also that it hasn't been tampered with.  If they're not equal, then something is wrong and Bill must mistrust the message.

The amount of confidence Bill has in the authenticity of the message depends on the amount of confidence Bill has that Alice has kept her private key secure.  If Bill is sure that Alice has kept her private key secure, then Bill can be sure the message came from Alice.  If malicious Mallory has gotten a copy of Alice's private key, then Mallory could have forged the message.

Notice that if Eve the eavesdropper can intercept a copy of the message, Eve can decrypt the digital signature with Alice's public key because it is, well, public.  However, it doesn't do Eve any good because one of the characteristics of that cryptographic hash is that the message cannot be reconstructed from the message digest.

Digital Certificates

In a previous post, I wrote about signing others' public keys as a way to improve our confidence that the key actually belongs to the person it claims to belong to. We saw earlier how a message could be both authenticated and protected from tampering through the use of a digital signature.  A public key can be protected in the same way.  When a public key is digitally signed, the result is called a digital certificate

The purpose of a digital certificate is to bind an identity to a public key.  When you created your public/private key pair, you put in your own email address, but you could have claimed to be or even  If you could get someone to use such a public key, thinking they were corresponding with the real Alice (or President) you'd be able to decrypt the messages because you have the corresponding private key.  A digital certificate helps us increase our confidence that a particular public key actually belongs to the claimed party.  It binds an identity to a public key.
The diagram at the left is a simplified representation of a digital certificate.  The part in green is all plain text.  It identifies as being the owner of the given public key.  This certificate has two digital signatures, one by Bob and one by Charlie.  By signing this certificate, Bob and Charlie are certifying that they have checked that the given public key actually belongs to

Here's how it works.  The certificate is just a message consisting of an identifier, a public key, and other information in plain text, the part shown as green in the diagram.  Each signature has two parts, identification of the signer, shown in yellow-green and stored as plain text, and an encrypted hash code, shown in red and encrypted with Bob's private key.  Someone who knows and trusts Bob can compute the hash code directly, using the part of the certificate shown in green, than use Bob's public key to decrypt the part shown in red under Bob's identifier.  If the computed hash code and the decrypted hash code match, we can have confidence that the part of the certificate in green hasn't been tampered with, and that the public key given really does belong to Alice.  How much confidence we have depends upon how much we trust Bob.  If we don't know or trust Bob, perhaps we know Charlie or someone else who has signed Alice's key.  The subject of trust was discussed in Using Encrypted Email.

A digital certificate can be signed by a number of individuals, creating a web of trust, or by a single trusted organization, called a certificate authority.  OpenPGP uses the web of trust model.

A Note on the Strength of Cryptography

Modern cryptosystems are very difficult to break.  If you used a 4,096-bit key size when you generated your key pair, cracking your encryption would take dozens of years using the best algorithms and fastest computers available.  Of course, Moore's law means that computers are getting faster, and in a dozen years, 4,096 bits may not be enough.  For now, even with a 2,048 bit key size, you are probably safe from any person or agency except possibly the NSA.

The weak points are the private key, stored on your computer and the passphrase, stored in your head. If an adversary can get both of those, your privacy is toast.  Without them , your privacy is protected against casual snoopers, and even persistent snoops like reporters.  You are in more danger from something like this than you are from a computational attack. Your private key backup is another weak point.  Guard it carefully.

While we're talking about the strength of cryptography it's time for a warning: don't roll your own.  While it might seem very cool to design and implement your own cryptosystem, it turns out to be surprisingly hard to do right.  Unless you have the equivalent of a Ph.D. in mathematics with an emphasis on cryptology, it's an extremely bad idea to trust important information to home brewed crypto.  Experiment all you want, but when you're serious about protecting information, use cryptosystems developed by experts, examined by other experts, and that have withstood the test of time.

Cryptography and the Law

There's a different question when it comes to law enforcement: can you be compelled to decrypt documents?  Even if law enforcement agencies cannot crack your encryption, they may be able to get a court to order you to decrypt documents yourself or to surrender your passphrase.  If you refuse, you might be held in jail until you change your mind.  Whether the Fifth Amendment protection against self-incrimination means you do not have to reveal your passphrase or the contents of encrypted documents has not been settled as of summer, 2013.  Different courts have ruled differently.  There is also a distinction at law between being forced to divulge your passphrase and being forced to produce documents in plain text (decrypted) form.

I'm not a lawyer, and cannot give legal advice.  Good, common sense advice is not to do illegal things nor possess illegal materials.  Encryption may delay an investigation, but it is unlikely to save you from the consequences of illegal actions.

With that said, I cannot emphasize too strongly that, in the United States, there is nothing illegal about using encryption. 

Previous article: Using Encrypted Email

Copyright © 2013 by Bob Brown

Creative Commons License
A Little About Encryption by Bob Brown is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Some of the symbols used in some of the illustrations were borrowed from a document on the subject of public key cryptography by Microsoft. Thanks, Microsoft!  The XKCD cartoons are used under the terms of the Creative Commons Attribution-NonCommercial 2.5 License.The quotation by Ken White is used by permission.

Using Encrypted Email

My people have no tradition of proofreading.  —Ken White
This is the second of three posts on encrypting email.  The others are It's Time to Encrypt Your Email and A Little About Encryption
If you've followed the directions in "It's Time to Encrypt Your Email," you have an email client that's set up to encrypt messages you send and decrypt messages you receive.  Now what?

Someone To Write To

Being the first person on your block to have public key cryptography for your email leaves you with no one with whom you can correspond securely. Let your frequent correspondents know about these articles and help them get started with public key cryptography for their email.  If you recruit two people to use encryption and each of them also recruits two more, soon everyone will be doing what we should be doing.

The Subject is Not Encrypted

OpenPGP and Enigmail  follow Internet standards, and those standards don't provide for encrypting the "header" information of email messages, including the subject lines.  This means that anyone who can eavesdrop on your mail stream can read your subject lines.  If your main reason for encryption is to foil casual snooping, just use subject lines as usual, but be aware that you are revealing information. Alternatively, you can use uninformative subjects like, "Encrypted message."  (An eavesdropper will be able to tell which messages are encrypted anyway, so you haven't revealed anything.)  For voluminous correspondence, where subject lines might be needed to categorize messages, and should not reveal information to eavesdroppers, you could use a code.  A subject of, "Spending weekend with Grandpa" might indicate one type of message, and "What about pizza?" another.

I've decided that, "Scroomall!  It's none of their damn' business anyway!" shall be my default subject line for encrypted mail.  Your mileage may vary.

Traffic Analysis

An adversary who can eavesdrop on your email can see with whom you correspond even if the contents of the messages remain hidden.  Sometimes inferences can be drawn from that information.  For example, if the owner of a successful private company suddenly starts communicating with investment bankers, one could speculate that the company is about to go public.  Traffic analysis countermeasures are far beyond the scope of this article, but users of encryption should be aware of the possibility that "they" know to whom you are writing, even though they can't read the messages.

It is increasingly the case that email systems use TLS/SSL encryption between client and server, and from one email server to another.  Although that makes traffic analysis more difficult, he header information of email messages is necessarily unencrypted while the message is "at rest," either on a server or at the origin or destination.  An adversary with access to an email provider's servers can read all of the header information: sender, recipients, subject, and the rest.  Of course, if the adversary can install monitoring software on the computer at the origin or destination, no part of the message is secure, even if the body is encrypted.

Digital Certificates

If Evil Eve the Eavesdropper could somehow replace Alice's public key with Eve's own, then Eve would be able to read messages intended for Alice.  We guard against that by embedding public keys in digital certificates.  A self-signed digital certificate is a digest (electronic fingerprint) of the public key and identifying data, computed using a cryptographic hash function and encrypted with the owner's private key.  Because public and private keys are cryptographic inverses of one another, anyone can compute the digest of Alice's digital certificate, then use Alice's public key (from the certificate) to decrypt the digest from the certificate, and finally compare the two.  In theory, if the two digests match, one should be able to trust the public key.  In practice, a self-signed digital certificate provides no significant protection against replacing both the public key and the encrypted fingerprint.  Self-signing does protect the certificate from tampering.

We overcome the problem that Eve could have generated a completely forged certificate by having the digital certificate signed with the private keys of one or more trusted third parties.  One approach is to use a single trusted party, the certificate authority.  The other approach, and the one employed by OpenPGP, is to have that key fingerprint signed by several third parties in whom we place varying degrees of trust.

A third party signs a certificate by computing the digest of the identifier and public key, then encrypting it with the signer's private key.  The signer's identity is added to the signature.   Anyone can find the signer's public key and check the digital signature as described above.

There is more about digests in A Little About Encryption.

 Key Signing

If a third party whom we trust has signed Alice's key, we can have a great deal of confidence that we really have Alice's key and not Eve's because changing the key would invalidate the signature.  If we don't know the person who signed Alice's key, we have a lot less trust.  Perhaps Eve has subverted the signer.

If several people have signed Alice's key, our confidence increases because Eve the Eavesdropper would have had to subvert each of them.  The more signers, the more confidence we can have, especially if one of the signers is known to us.  This creates the "web of trust" that lets us have confidence that the public keys of others have not been tampered with, and does so without the need for a central authority.

You may be asked to "sign" someone's key, or you may want to ask someone to sign yours.  To have a friend sign your key, approach the friend in person, with your name or email address and key fingerprint written on paper.  If your friend agrees to sign your key,your friend will take the paper, obtain your public key from a key server, and verify that the key fingerprint you supplied on paper patches that of the key from the key server.  If the two fingerprints match, your friend will get your key from a key server, "certify" the key as belonging to you by adding his digital signature, and send it back to the key server.

There's no need to go wild, but your key will be more trusted if at least four or five people have signed it; more is better. Try to have it signed by people who move in different circles; that will increase the probability the someone who needs your key will know someone who has signed it, or "know someone who knows someone."

A Key Signing Party 

 Since getting people who can verify each others' identity to sign each others' public keys is a Good Thing, and since socializing with like-minded people is also a Good Thing, you might want to host (or instigate) a key signing party.  You won't even need access to computers at the party.  The host prepares a list of attendees and information about their public keys and makes a copy for everyone at the party.  Attendees bring their own key information and picture IDs to the party.  Each guest checks the information for each other guest, and then later signs the keys of those whose information checks out.  The "later" part is why you do not need, and should not have, computers at the party.

There are many sets of instructions for organizing key signing parties on the web.  The best one I've found so far is the W4KWH Key Signing Party Guide.

If you host or attend a key signing party, be sure you scrupulously verify the identification and key information of everyone whose key you intend to sign.  Do not make this mistake:

Trusting Others' Certifications

Suppose Alice wants to communicate securely with Bob. She retrieves Bob's public key from a key server and  finds that it has been signed by Charles, David, Frank, and George.  Can we say more about how certain Alice can be that she really has Bob's public key than, "signed by four people?"  Yes, especially if Alice knows any of those people, or knows anyone who has signed their keys..

Alice (and you) can assign a level of trust to every key on her keyring of public keys.  This is a subjective decision; Alice (and you) get to decide.  The levels of trust are:
  1. Unknown
  2. Untrusted (known to sign public keys carelessly or maliciously; see the cartoon!)
  3. Marginally trusted (probably careful when signing public keys)
  4. Fully trusted.
Anything signed with Alice's own private key is assumed to be fully trusted; everything else is initially set to "unknown."  Alice can change the trust level of keys on her keyring.  If Charles's key is on Alice's keyring, she can set its trust to "marginally trusted" or even "fully trusted."

Given the idea of levels of trust, one can apply an algorithm to help Alice decide whether the key she has retrieved from a key server for Bob is really his.  A key is valid (i.e. trustworthy) if two conditions hold.  First, it must have been signed by "enough" trusted keys.  "Enough" means:
  1. Alice has signed it herself, or
  2. It has been signed with at least one key that is fully trusted by Alice, or
  3. It has been signed with at least three keys marginally trusted by Alice.
The second condition is that the path of signatures from Bob's key to Alice's has five or fewer steps.  That may seem very restrictive until one considers the concept of six degrees of separation.

Those are the default parameters for GnuPG; you can change the parameters to be more or less restrictive.  Levels of trust are private, and are not shared or exported when you sign someone else's public key.

Creating a Revocation Certificate 

Once a key has been uploaded to a key server, it is effectively impossible to remove.  Yet, you might want to remove your key for a number of reasons.  It might be something as scary as having your private key compromised, or as mundane as getting a new email address, which is effectively a new identity.  (You can also deal with a new email address using subkeys, which are explained in the Gnu Privacy Handbook.  Subkeys are really the right way to deal with new addresses.) Although you can't remove a key from the key servers, you can revoke it... if you have a revocation certificate. Revoking your key tells everyone it is no longer effective to use that public key.

The revocation certificate must be signed using your private key.  One of the reasons you might want to revoke a public key is that you no longer have access to the corresponding private key.  As I wrote in an earlier post, that can happen if you forget your passphrase or have a disk failure.  So, generate that revocation certificate soon after you generate your key pair.  Once you've generated a revocation certificate, do not import it.  You will do that only if you're ready to revoke the public key.  Instead, store it someplace safe.  Remember, anyone who can get access to your revocation certificate can revoke your public key.

You will need to use the command interface of GnuPG to create your revocation certificate.  Open a command window and navigate to the directory where gpg.exe is stored.  On my computer, that's
\Program Files\GNU\GnuPG.
The command line to enter is
gpg --output <key ID>.revoke.asc --gen-revoke <key ID>
Where <key ID> represents the eight hex character ID of the key for which you want to create a revocation certificate.  Here, edited slightly, is what it looked like when I created a revocation certificate for key ID 375F8696.  What I typed is in bold.

gpg --output 375f8696.revoke.asc --gen-revoke 375f8696
sec  2048R/375F8696 2013-04-07 Bob Brown <my_email>
Create a revocation certificate for this key? (y/N) y
Please select the reason for the revocation:
  0 = No reason specified
  1 = Key has been compromised
  2 = Key is superseded
  3 = Key is no longer used
  Q = Cancel
  (Probably you want to select 1 here)
  Your decision? 1
  Enter an optional description; end with empty line:
  Reason for revocation: Key has been compromised
  (No description given)
  Is this okay? (y/N) y
You need a passphrase to unlock the secret key for
  user: "Bob Brown <my_email>"
  passphrase entered here
  2048-bit RSA key, ID 375F8696, created 2013-04-07
  ASCII armored output forced.
  Revocation certificate created.

The result will be a file called <key ID>.revoke.asc; in the example, 375F8696.revoke.asc, in the same directory where you ran the command.  Do not leave it there!  You need the revocation certificate if you lose access to your private key, which could happen if your computer is stolen or your hard disk fails.

These instructions came from the Gnu Privacy Handbook.  I recommend having a good look at it to see what else one can do with

Signing Unencrypted Messages

GnuPG can be used to add a digital signature even to messages that haven't been encrypted.  Such a digital signature makes the messages tamper-evident and provides for cryptographic authentication of the sender.  It also adds PGP headers and a base 64 encoded digital signature to the body of the message, making one's messages somewhat ugly.  There are two reasons for signing unencrypted messages.  One, of course is to provide tamper-resistent, authenticated messages.  The other, and possibly the more important one, is for it to cause recipients of such messages to say, "WTF?" and thereby open a discussion of why they, too, should be encrypting their email.  For that second reason alone, it may be worth it.

Validating (Verifying) Signed Messages

Enigmail can be configured to add digital signatures to all messages by default, even those that are not encrypted.  It can also be configured on a per-address basis.  Consider adding a digital signature to all of your email, or at least to mail sent to friends who might be amenable to encrypting their own email.

The cartoon is a little tongue-in-cheek... well, a lot tongue in cheek.  Validation of signatures is automagic if the recipient is using an email client that understands OpenPGP.  Such a client may even conceal the headers and just give a green bar or other indication that the message has been authenticated.  Someone using an email client that doesn't understand OpenPGP can still validate digital signatures.  To do so, copy the entire message, beginning with "-----BEGIN PGP SIGNED MESSAGE-----" and ending with "-----END PGP SIGNATURE-----" and save it to a temporary file.  It can then be validated using the Decrypt/Verify File function of Kleopatra.  (It's under the File menu.)  The recipient must have imported the sender's public key certificate.

Thunderbird Portable Edition

My laptop goes where I go, so I can deal with encrypted email just about anywhere.  If you primarily use a desktop computer and set up encryption there, it might be difficult to read or send encrypted email from elsewhere.  One solution might be Thunderbird Portable Edition.  This is Thunderbird bundled with the PortableApps launcher so that it will run from a USB flash drive.  One can install GPG and Enigmail on the same flash drive.  Using Thunderbird this way leaves no information behind on the host computer.  The Windows operating system is required.

If you set up and use Thunderbird Portable Edition, your private key will be on that flash drive; be sure to guard it well.  Consider using an encrypted flash drive.

If you use any of the PortableApps applications, please consider making a donation or buying one of their flash drives.  The software is free; the effort of making it portable is supported by donations.

Previous article: It's Time to Encrypt Your Email    Next article:A Little About Encryption

Copyright © 2013 by Bob Brown 

Creative Commons License
Using Encrypted Email by Bob Brown is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

The XKCD cartoons are used under the terms of the Creative Commons Attribution-NonCommercial 2.5 License.The quotation by Ken White is used by permission.

Thursday, May 2, 2013

It's Time to Encrypt Your Email

My people have no tradition of proofreading.  —Ken White

Too long; didn't read

I set out to write some simple instructions on setting up encrypted email.   Being a college teacher, I couldn't resist answering the "because" question for every step.  I also wandered into the ways one could use encrypted email and the other features available.  The article became way too long for anyone to actually use, so  I've broken it up into three parts.  This is the first of the three.  The other two are:

Why Encrypt?

We've read about "hackers" and reporters breaking into peoples' email. The FBI is back again with a proposal that would make "wiretapping" electronic communications easier. It's probably already too easy. If you're not sure, ask David Petraeus. Current (2013) law provides very little protection for email messages and almost none for messages left on your mail provider's server for longer than 180 days.  According to the Electronic Frontier Foundation, some of the biggest providers of email (Apple, AT&T, Comcast, Verizon, and Yahoo) do not require warrants for access to "content," which appears to include email.  (2013 data.)  In the summer of 2013, we found that snooping by the NSA in particular is far more widespread than most of us had imagined.

Most of the personal email most of us write is completely innocent, things like deciding on where to go for pizza, but even innocent email can contain confidential information.  I needed the Social Security numbers of my nieces and nephew to make them beneficiaries in case of my death.  Even though those exchanges were innocent, they contained information others should not have. I have recently been reminded that anything you communicate to your lawyer should be encrypted to protect the attorney-client privilege.

As you can see, there are good reasons for keeping your email confidential.  The best one is that the contents of your email are none of their damn' business!  whoever "they" may be. You can protect your personal email by encrypting it, and it's easy to do.  Read on to find one way to do it.

Following these instructions will allow you to encrypt mail that you want encrypted, and all the software you need is free.  Messages that you don't want encrypted will not be.  That's important because many of the people with whom you correspond will not have encryption set up at "their end."   Tell the ones who don't about this article!

Two Ways to Encrypt Email

Email can be encrypted using one of two standards: S/MIME or OpenPGP, that are not interoperable.  The Thunderbird email client discussed below is ready for S/MIME out of the box.  Trouble is, that requires a digital certificate for which you might have to pay.  These instructions are for OpenPGP, which is based on a "web of trust" rather than on digital certificates.  (If you have friends who use S/MIME encryption, Comodo offers free certificates with a one-year expiration period.  You can install both S/MIME and OpenPGP encryption, but you have to keep track of who uses what.)

Note for Users of  MacOS 

The instructions below will work for MacOS, but after I wrote this, the folks who develop GPG Tools for Mac released Version 2 of GPG Mail.  I don't have access to a Mac, so I haven't tested this personally, but a colleague with a Mac says it just snaps in and works.  If you use the Apple Mail application, you can probably just install GPG Mail instead of following my outline.  Before you start, please do read about pass phrases, key generation, and making a backup of your private key, below; that information is applicable to configuring GPG Mail.  You should actually read the whole article, and the two that follow.  If you try using GPG Mail, please leave a comment reporting your experience.  (Note added 2013-08-11.)

What We are Going to Do

These instructions are primarily for Windows users, but the software described will run on Linux and Mac OS as well.  This may seem long and complicated, but if you take it step-by-step, it'll be easy.  I recommend reading through the entire post before you start.  Here's what we're going to do:
  1. Install a "local" email client, Mozilla Thunderbird, on your computer.
  2. Install public key cryptography software, either GPG4Win or Gnu Privacy Guard, on your computer.
  3. Devise a passphrase to protect your private key.
  4. Generate your own public/private key pair.
  5. Install the Enigmail plugin.
  6. Upload your public key to a keyserver.
  7. Make a backup of your private key.
Optionally, you may want to create a revocation certificate and have your public key signed by a few other people.  There's more on that in Using Encrypted Email.

Install a Local Email Client

To start with, you will need an email client that runs on your own computer.  If you are already running Thunderbird, or perhaps Seamonkey, you're done with this step.

Otherwise, download and install Mozilla Thunderbird.  In most cases, Thunderbird can configure itself automatically with just your email address because it has a database of settings for major email providers.  If, for some reason, automatic configuration fails, you will need, from your email provider, the server addresses and port numbers to configure IMAP or POP incoming mail and SMTP outgoing mail.  These should be available from your email provider's support site.  For example, the settings for Google's gmail, are, port 993 and, port 465.  You will also need to know the connection type, which will often be SSL/TLS.

Install GPG4Win or Gnu Privacy Guard

Windows users, download and install GPG4Win.  Linux users should install Gnu Privacy Guard, and Mac OS users should install GPG Tools.

Part of setting up this software involves generating a public and private key for yourself.  Be sure you've read the next sections before you do that.

Devise a Passphrase

It is important that you keep your private key private.  You will use a "password" for that.  Your encryption is no stronger than this word or phrase. Please don't use any of the top 10,000 passwords.  I recommend using a phrase you will remember, something like, "My best times are those I spend at the beach."  That is very easy to remember, but it would be difficult even for someone who knows me well to guess it.  The phrase should be more than a couple of dozen letters to foil automatic guessing, but you are going to have to type this phrase every time you encrypt or decrypt a message, so don't write a book!

The people who write password cracking software are onto the rules of grammar.  For a little more strength, rephrase as Yoda-speak: "Those I spend at the beach my best times are."

For the strongest passphrase, generate it with Diceware.  If you want to experiment with Diceware, I have a set of electronic dice.  Get some real dice to generate your actual passphrase if you choose this method.

If you forget your passphrase, you're toast.  Write it on a slip of paper and put it in a book you use infrequently. When (if ever) you are absolutely sure you have it memorized, tear that slip of paper into tiny pieces and flush them.

Generate a Key Pair

Once you've settled on a passphrase, you are ready to generate your public and private keys.  For Windows, you do this with Kleopatra, which is part of GPG4Win.  For reasons that don't need to be explained here, Kleopatra, and possibly some of the other software, talks about "certificates" in reference to public and private keys.  In Kleopatra, choose File → New Certificate.  Generate an OpenPGP key.  Make the key length at least 2,048 bits. You set the key length using the "Advanced Settings" button after you've started to generate an OpenPGP key.  Choosing the maximum size, currently 4,096 bits for GPG4Win and GPG Tools, is good.

Install the Enigmail Plugin

Download and install the Enigmail plugin for Thunderbird.  Enigmail is the "glue" that connects the encryption software to Thunderbird.  It adds toolbars for encrypting and decrypting email.  You'll need to pick your operating system platform and the version of Thunderbird you're using.  You can find the Thunderbird version through Help →About.

Upload Your Public Key to a Keyserver

To make your public key as public as possible, upload it to a keyserver.  The keyservers talk to each other, so uploading to one of them is generally enough. Use Settings → Configure Kleopatra → Directory Services and click Add to add a keyserver.  The default is, and accepting the default is generally all you need to do.  Once a keyserver has been configured, choose File → Export Certificates to Server.  Ignore (for now) the warning about creating a revocation certificate.

Make a Backup of Your Private Key

Earlier I wrote that if you forget your passphrase, you're toast.  The same thing is true if you lose your private key.  If you've followed these instructions, your private key is stored on your computer.  A disk crash or a new computer might mean you can no longer decrypt mail that others send to you.  To prevent that, make a backup of your private key now and any time you make a change.

To do that with Kleopatra, select our own certificate (key) and select File → Export Private Certificate. (Some versions say, Export Secret Keys) Provide a file name, like bbrown_private_key, and do not check ASCII Armored.  Copy the backup to a flash drive or burn it to CD and guard it carefully; it is not protected by your passphrase.

Exporting your private key also exports your public key.  If you ever need the backup, you will be able to import it with Kleopatra.

What About Web Mail, Tablets, Smart Phones?

To be secure, encryption and decryption necessarily has to take place on the "end device," your computer, tablet, or phone.  If it happened at your email provider's server, your email provider would have to have your key.  If they have your key, they can divulge the contents of your email, perhaps accidentally or perhaps under a secret court order.

If you were using a web mail client, it will still work, but you won't be able to send or read encrypted mail. The post on Using Encrypted Email includes a brief discussion of Thunderbird Portable Edition, which will let you use encryption with others' computers.

For phones and tablets, there is software for Android that will let you use encryption, and software for iOS that will allow reading, but not creating, encrypted messages. A new iOS app seems to provide full encryption and decryption.  I've successfully installed APG and K-9 on my Android tablet and can send and receive encrypted mail.  I'll try to write some instructions presently.

About Your Work Email

Even if you work for a very permissive organization, they probably wouldn't appreciate your adding encryption to your work email.  Beyond that, free email accounts are readily available for personal use.  Mixing personal and work email can cause you serious problems.  For some examples, search on "fired because of email."

Some Terminology

These definitions may help you navigate the documentation of the programs with which you will be working.
OpenPGP is a standard describing a mechanism for both encrypting and digitally signing files. Those files may be email messages or a "plain" data file. There is no "OpenPGP" program; two programs that implement the OpenPGP standard are described below.
PGP was a company, since acquired by Symantec, and also the name of that company's products. The PGP products implement the OpenPGP standard. They're commercial products; they cost money. People pay Symantec money to get technical support, regular product upgrades, etc. If you are installing encryption for a company, and not for personal use, consider the Symantec products or those offerd by other companies.  There was a free version of PGP, but it is now very out of date and should not be used.
GnuPG, also called Gnu Privacy Guard or GPG, is a free and open-source implementation of OpenPGP. As with other free software, support consists only of forums, mailing lists, and web articles. Upgrades and fixes are contributed by a dedicated group of volunteers.
GnuPG refers to your public key and those of others as certificates because that's what they are. What's produced is a public key with a digital signature signed with the corresponding private key. That's a self-signed digital certificate. While it doesn't provide any assurance of correct binding to an identity, it does provide protection against tampering.

Ready for More Information? 

Now that you have encryption set up, read Using Encrypted Email.  For an overview of how this all works, try A Little About Encryption.

Did You Do This?

Every encrypted message is a tiny protest against the government's massive surveillance apparatus.  Please encourage others by leaving a comment below.  It can be as short as "I did it!" or as long as a description of your experience, but you really will help others.  Please spread the word by sharing this post.  Linking is easiest and lets me make updates, but the Creative Commons license lets you copy the entire post, too.

Copyright © 2013 by Bob Brown 
 Creative Commons License
It's Time to Encrypt Your Email by Bob Brown is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

The quotation by Ken White is used by permission.

Monday, January 21, 2013

DenyHosts Remove / Delete an IP Address

If you got here from a search, it's because you're looking for a way to remove an IP address from the tables used by the DenyHosts program.  Read on!

DenyHosts is a nifty Python program that monitors Unix/Linux log files for failed SSH logins and, after a configurable number of attempts, adds the source IP address to /etc/hosts.deny.  It's a simple but effective intrusion prevention system for SSH servers.

I'm a college teacher and I operate a server that my students use via SSH.  I run DenyHosts to stop SSH hacking, which happens all the time from all over the world.  At the beginning of each semester, it's not unusual for several students to lock themselves out due to multiple failed login attempts.  The DenyHosts FAQ gives instructions for removing addresses that should not have been added.  It's a fiddly, time-consuming process, and I wanted to automate it.  I wanted to be able to type:
and have the address removed from all the places from which it should be removed.  (The IP address in the example is an obvious fake for obvious reasons!)

Here is a bash script that will do that.

The Script

echo Removing $1 from denyhosts tables
IP=`echo $1 | sed 's/\./\\\\./g'`
service denyhosts stop
eval "sed -i /$IP/d /etc/hosts.deny"
eval "sed -i /$IP/d ${WORK_DIR}hosts"
eval "sed -i /$IP/d ${WORK_DIR}hosts-restricted"
eval "sed -i /$IP/d ${WORK_DIR}hosts-root"
eval "sed -i /$IP/d ${WORK_DIR}hosts-valid"
eval "sed -i /$IP/d ${WORK_DIR}users-hosts"
echo $1 >> ${WORK_DIR}allowed-hosts
service denyhosts start

That's it; copy and paste this into a file in the root user's space, make it executable, and you're good to go!  But before you copy-paste it (which you are encouraged to do) please read my notes below and any comments there may be.

The Notes

First let me say that I do not spend my days writing bash shell scripts. I am absolutely, positively certain this can be improved upon.  I'm certain it's not bullet-proof, and it's probably not even bullet-resistant.

If you can help by improving on it, please make comments below.  You'll help me and everyone who finds this page and considers using this script.

There's no error checking in what's above.  I'm not even sure what should be checked, much less how to go about checking it.  Deletion of lines will fail silently if the script is given a bogus IP address or a correctly-formed address that doesn't exist in the files edited.  However, it will add whatever is given as an argument to the end of the allowed_hosts file; That's probably not what is wanted if the argument to the script is bad.

There should probably at least be a check for the argument being empty because that'll create an empty line in the allowed-hosts file. Ideally, one would validate that the argument was four numbers 0-255 separated by periods.

One could go even further and exclude 127.0.0.* and any IP address with a zero first octet.  There may be other things that should be excluded.

There may be a way to set WORK_DIR for a specific installation.  I used the default, and so the script will have to be edited for those who did not.

How it Works

Most things should be pretty obvious.  The fourth line, where $IP is defined, escapes the periods in the IP address so the address can be used in a regular expression.  The lines that begin with "eval" delete any line with the given IP address from the file specified. The -i switch instructs sed to write the edited file back to the input.  I understand that not all versions of sed support -i.  Be sure to test.

Beware! The second-from-last line:
echo $1 >> ${WORK_DIR}allowed-hosts
appends the given IP address to the allowed-hosts file.  This will keep it from being blacklisted again.  That may not be what you want to do in general, and it's certainly not what you want to do if you run this with an invalid IP address. Remove or comment that line of you want to allow an address to be listed again, and be sure to edit the allowed-hosts file if you accidentally run the script with an invalid address.

Saturday, July 14, 2012

Six Simple Rules for Secure Storage of Passwords

A few weeks ago, password files from LinkedIn,, and eHarmony were released by crackers. Yesterday it was Yahoo and today it's nVidia and Billabong, an Australian clothing retailer. The Yahoo and maybe Billabong passwords were stored as plain text, a particularly egregious mistake.

In (a tiny bit of) defense of the yahoos at Yahoo, the file that was stolen and released came from a system not developed at Yahoo. Yahoo bought Associated Content and renamed it Yahoo Voices. Apparently no one at Yahoo thought to do a security audit of the systems they acquired when they bought Associated Content.

Unhappily for over 400,000 people, Associated Content and later Yahoo used email addresses for user IDs.  Other organizations do this too.  All the time.  In at least some cases, it is likely that the hapless subscriber used the same password for Yahoo as for the email account.  Oops!

Securing passwords is not rocket science. Here are six simple rules for securing stored passwords. Toward the end, I'll give you a seventh rule that might have saved Yahoo's bacon even though they violated rules two through six and maybe rule one.
  1. Plain text passwords must be transmitted from user to host over an encrypted connection, e.g. SSL/TLS.
  2. No password will ever be stored in plain text; all passwords will be hashed. 
  3. The password hash will use a salt of non-trivial length, e.g. 128 bits. 
  4. Each password must have a different, random salt. 
  5. The hash function must be resistant to collision attacks. 
  6. The hash function must be computationally expensive and difficult to parallelize. 
Let's look at each one of these in turn.

Plain text passwords must be transmitted from user to host over an encrypted connection

This one ought to make sense right on its face.  If someone can install a network sniffer between a user's web browser and the the server they're trying to talk to, that someone can capture passwords unless they're encrypted.   I've never worried much about the garden variety hacker capturing passwords with a sniffer.  Unless they could get close either to my computer or the the server I want to talk to, it would be like drinking from a fire hose.  However, if the FBI, CIA, NSA, and/or Sheriff are after you, they can probably do it.  The same is likely to be true in certain other countries. Using an encrypted connection makes it much harder to snoop on passwords as they're sent to the server.

No password will ever be stored in plain text; all passwords will be hashed

"Hashed" means  that when we set up the account for someone on our server, we transform the password into gibberish, or "hash."  To produce the hash, we want to use a function that cannot be undone.  So, given the hash value, there is no way to "decrypt" it to recover the original password.  If an attacker succeeds in capturing the password file, he or she still cannot undo the hash; the passwords are safe.  (Not.)

After an account is set up with the hashed password, when our user wants to log in, we just run the hash function using the password supplied with the login attempt.  If the result of hash function gives the same value that was stored when the account was set up, we believe that the password given matches that when the account was set up, and allow access. (But, see the section on collision resistance below.)

The password hash will use a salt of non-trivial length

Although you cannot undo a password hash to recover passwords from the hashed file,  it isn't really necessary to do so.  An attacker can take a large electronic dictionary plus a list of commonly used passwords and run the whole list through the hash function. The attacker now has a list of dictionary words and their hashes.  If a hash from the stolen password file matches a hash in the attacker's list, the attacker has determined the password, and done so without inverting (or "decrypting") the hash.  How does an attacker know which hash algorithm to use?  Kerckhoffs' Principle says that, to be secure, we must assume that an attacker knows the system.  A determined attacker can try many hash algorithms.  As soon as one produces a hit, the attacker knows that's the one in use.

To defend against the use of dictionaries of plain text words and their hashed equivalents, we add some random bits to the password before hashing it.  Cryptographers call this a "salt."  If we just add a zero or a one bit, now an attacker must compute the hashed dictionary twice, once for the case of zero and once for the case of one.  That doesn't help much, but if we add 12 random bits, the attacker needs 4,096 encrypted dictionaries.  Adding 32 random bits to the password before hashing it means the hacker has to make 232 or more than four billion encrypted dictionaries.  With 128 bits, the attacker would need 2128 encrypted dictionaries.  That's a Very Large Number Indeed.  (It's about 3.4×1038.)  Pre-computing that many copies of a hashed dictionary may be theoretically impossible because of the amount of energy required for the computation.

Of course, we have to store the salt someplace, and if an attacker can get the password file, the attacker can probably get the salt, too.  That leads to the next rule.

Each password must have a different, random salt

Instead of just one salt for everyone, use a different, random value for each user. You only have to calculate it once, and you store it with the password hash.   By making the salt very large (see above), a system could have millions, or even hundreds of millions, of users but the chance that two users would get the same random salt remains very small.  Now the attacker has to hash the dictionary file for every user password to be tested.  The problem just got significantly harder.

Each salt must be as close as we can get to truly random; if you use something like user ID for the salt, an adversary can predict it.   Such random numbers are generated using an algorithm called a cryptographically secure pseudo-random number generator.

A second salt, called a "key," used system-wide and not stored with the hashed password adds still more work for the bad guys.  Such a scheme is called a "keyed hash." One place to store the key is as a constant in the programs that process passwords.

The hash function must be resistant to collision attacks

A hash collision occurs when two different passwords generate the same hash.  For example, if "icecream" and "darkbeer" both generate the hash "etaionshrdlu" then either password can be used to log on to the account which has that hash. (By the way, both of those are pretty miserable passwords.)  The salt helps, but we still need to avoid collisions.  If "darkbeer" isn't in the cracker's dictionary, but "icecream" is, they've still cracked your password!

Happily, cryptographers know all about collision attacks, and so they design hash functions that are resistant to such attacks.  Perhaps now is a good time to caution you against "roll your own" cryptography.  It turns out to be surprisingly hard to do right, and you don't need to.  There is software that will do what you need, freely available.

The hash function must be computationally expensive and difficult to parallelize

We should be more worried about criminals cracking passwords than about "hackers" cracking passwords.  Willie Sutton robbed banks because "that's where the money is."  Criminals want to steal your login credentials for the same reason, and some of them can devote impressive resources to the task.  When we say that the hash function must be computationally expensive, we only mean that it takes the computer a long time to turn the plain text password into the hash code.  You won't care, and probably won't even notice, if it takes a quarter second longer to log in to some server.  However, if each try at password guessing takes a quarter second, that's four guesses per second for the attacker, instead of potentially billions of guesses per second.

Graphics cards with multiple processors are available and cheap.  If an adversary can spread computation among multiple processors, then "computationally expensive" might no longer mean "slow."  So, the hash function should be designed so that it is difficult or impossible to divide the work among multiple processors.

About users and passwords

Your users will pick terrible passwords.  No amount of programming can completely prevent that.  Attackers know this, and they'll try common passwords even if you've made a brute force attack too difficult.  For some fun reading, type 100 most common passwords into your favorite search engine and have a look at a few of the lists. (Um, and if you find any of your passwords on any of the lists, change 'em right now!)

You can help users pick good passwords by providing some guidance and, most especially, providing for very long pass phrases.  To see what I mean, check this cartoon from XKCD.  Suggest that, when people register on your site, that they pick a pass phrase rather than a password, and maybe even link to XKCD.

Some security experts will disagree with me, but I think trying to force particular combinations, like "one uppercase letter, one number, one punctuation character" makes matters worse rather than better.  It encourages people to re-use passwords or to write them down.  Writing passwords down isn't necessarily bad in and of itself, but it provides an opportunity for something else to go wrong.

About these six simple rules

Following these six simple rules will make it much harder for attackers who steal your password file to recover plain text passwords.  Following the rules does not make guessing impossible, so if there's a breach, you have to notify people.  With luck, the things you've done will provide time for you to notify your subscribers in case of a breach and for them to change their passwords elsewhere.  They'll also keep people from saying (about us) "How could they be so stupid?"

None of these things will prevent your password file from getting stolen in the first place.  That's a subject for another day, but see below for one suggestion.  None of these things will save the user who chooses "keepout" for a password.

One last thing about the simple rules above.  Attackers can't easily recover a password even if they have the password file.  Neither can you!  So, you'll have to provide a mechanism for the hapless person who forgets a password to reset it.  There is no way to tell them what it was.  Another subject for another day is how to keep an adversary from subverting the password reset process, but it's something to worry about.

Password software

There is time-tested software, developed by real cryptographers, that will help you store passwords in a way that meets the last five tests.  (You have to use SSL or TLS to meet the first one.) I'm not going to include links because there's too much information, but if you're storing passwords, look into bcrypt, scrypt, or PBKDF2.  This really isn't rocket science.  One just needs to understand the need to do it.

The seventh rule: Parameterized queries

The Yahoo password file was stolen through an SQL injection attack.  It seems possible that others were stolen in the same way.  SQL injection is also a subject for another day, but one bit of advice will stop SQL injection attacks cold: use parameterized queries for all database operations.

May password file compromise never happen to you. 

Saturday, January 7, 2012

Total Defense, Anti-Virus, Etc.

I gave up paid computer consulting a year or so ago when the last company I was working with was sold.   I still do some unpaid consulting, including for a doctor who gives away more medical care than he charges for.  Some years ago I bought, with his money, a Computer Associates (CA) perpetual license for their antivirus product.  All was well for five years.  Yesterday I uninstalled one license from one of the practice's three computers to move it to a new one.  When I installed it on the new computer, it immediately reported that it was expired!

Well, I'm careful when I spend other people's money, so I had the order number, license keys, etc. all at hand. "This won't be a problem," says I, "I'll just call customer service and they'll tell me how to fix it."

Didn't work. I reached a woman with a barely-understandable Indian accent who pretended not to understand the meaning of "perpetual." When I insisted that they look up the original order using the order number I had right in front of me, I was told that technical support could not do that, but they'd transfer me to customer service. OK. There was an interminable wait, then a man with Yet Another thick Indian accent insisted over and over that I "just have to pay the renewal fee." This man also pretended not to understand "perpetual," even after looking up the order. He also completely ignored my many requests to talk with a supervisor. I finally, after 23 minutes of this, hung up in frustration rather than saying what I thought of this treatment, which, of course, is exactly what they wanted me to do. (Yes, I am tough enough to be polite even in circumstances like this one.)

Bottom line: the people at Total Defense, "a former division of CA" will not help you when things go wrong.

The good news is that Microsoft Security Essentials seems, from all reviews, to be an effective AV product. It has had time to mature. (It was introduced in fall of 2009.) And it's free for home and small business use! It works with 32 bit XP and both 32 and 64 bit Windows 7. 

Bye, Total Defense! See you around!