Hacking Gmail

Hacking Gmail™ Ben Hammersley 01_59611x ffirs.qxp 11/28/05 11:09 PM Page iii Hacking Gmail™ 01_59611x ffirs.qxp 11/28/05 11:09 PM Page i 01_59611x ffirs.qxp 11/28/05 11:09 PM Page ii Hacking Gmail™ Ben Hammersley 01_59611x ffirs.qxp 11/28/05 11:09 PM Page iii For general information on our other products and services or to obtain technical support, please contact our Customer Care Department within the U.S. at (800) 762-2974, outside the U.S. at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging-in-Publication Data Hammersley, Ben. Hacking Gmail / Ben Hammersley. p. cm. Includes index. ISBN-13: 978-0-7645-9611-7 (paper/website) ISBN-10: 0-7645-9611-X (paper/website) 1. Gmail (Electronic resource) 2. Electronic mail systems. 3. Internet programming. I. Title. TK5105.74.G55H36 2006 004.692—dc22 2005029719 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. ExtremeTech and the ExtremeTech logo are trademarks of Ziff Davis Publishing Holdings, Inc. Used under license. All rights reserved. Gmail is a trademark of Google, Inc. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Hacking Gmail™ Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2006 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN-13: 978-0-7645-9611-7 ISBN-10: 0-7645-9611-X Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 1B/RU/RS/QV/IN No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Website is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Website may provide or recommendations it may make. Further, readers should be aware that Internet Websites listed in this work may have changed or disappeared between when this work was written and when it is read. 01_59611x ffirs.qxp 11/28/05 11:09 PM Page iv Meanwhile, back in Florence, Anna, Lucy, Mischa, and Pico ignored the swearing and kept me fed. Love, as ever, to them. 01_59611x ffirs.qxp 11/28/05 11:09 PM Page v 01_59611x ffirs.qxp 11/28/05 11:09 PM Page vi About the Author Armed only with a PowerBook and some fine pipe tobacco, Ben Hammersley is a journalist, writer, explorer, and an errant developer and explainer of semantic web technology. He’s also liable to spread his dirty, dirty words over at The Guardian. As an Englishman of the clichéd sort, Ben’s angle brackets always balance, and his tweed is always pressed. He’s not worn trousers for six months now. Ask him about it sometime. 01_59611x ffirs.qxp 11/28/05 11:09 PM Page vii 01_59611x ffirs.qxp 11/28/05 11:09 PM Page viii Credits Executive Editor Chris Webb Development Editor Brian Herrmann Technical Editor Justin Blanton Production Editor Kenyon Brown Copy Editor Nancy Rapoport Editorial Manager Mary Beth Wakefield Production Manager Tim Tate Vice President and Executive Group Publisher Richard Swadley Vice President and Executive Publisher Joseph B. Wikert Project Coordinator Michael Kruzil Graphics and Production Specialists Carrie A. Foster Lynsey Osborn Melanee Prendergast Quality Control Technicians Leeann Harney, Jessica Kramer, Charles Spencer, Brian H. Walls Proofreading and Indexing TECHBOOKS Production Services Cover Design Anthony Bunyan 01_59611x ffirs.qxp 11/28/05 11:09 PM Page ix Contents at a Glance Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Part I: Starting to Use Gmail . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1: Desktop Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Chapter 2: Integrating Your Existing Mail. . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 3: Gmail Power Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Part II: Getting Inside Gmail . . . . . . . . . . . . . . . . . . . . . . . . . 27 Chapter 4: Skinning Gmail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Chapter 5: How Gmail Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Chapter 6: Gmail and Greasemonkey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Chapter 7: Gmail Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Chapter 8: Checking for Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Chapter 9: Reading Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Chapter 10: Sending Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Part III: Conquering Gmail . . . . . . . . . . . . . . . . . . . . . . . . . 167 Chapter 11: Dealing with Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Chapter 12: Addressing Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Chapter 13: Building an API from the HTML-Only Version of Gmail. . . . . . . . . . 183 Chapter 14: Exporting Your Mail. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Chapter 15: Using Gmail to . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Chapter 16: Using GmailFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Appendix: Long Code Listings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 02_59611x ftoc.qxp 11/28/05 11:05 PM Page x Contents Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Part I: Starting to Use Gmail 1 Chapter 1: Desktop Integration . . . . . . . . . . . . . . . . . . . . . . . . 3 New Mail Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Mac OS X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Linux, etc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Redirecting mailto: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Multiplatform/Mozilla . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 OS X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 GmailerXP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 2: Integrating Your Existing Mail . . . . . . . . . . . . . . . . . . 11 Importing Your Mail into Gmail . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Gmail Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Setting Up Pop Access Inside Gmail . . . . . . . . . . . . . . . . . . . . . . . . . 12 Setting Up Pop Within an Application . . . . . . . . . . . . . . . . . . . . . . . . 13 IMAP for Gmail. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 And Now . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 3: Gmail Power Tips. . . . . . . . . . . . . . . . . . . . . . . . . 15 Keyboard Shortcuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Plus Addressing and Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Other Addressing Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Quickly Mark a Group of E-Mails . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Send Executables as Attachments . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Advanced Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 And Now . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 02_59611x ftoc.qxp 11/28/05 11:05 PM Page xi Part II: Getting Inside Gmail 27 Chapter 4: Skinning Gmail . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Deconstructing Gmail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 The Top Section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 The Navigation Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 The Activity Area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 The Bottom Section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Applying a New Style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Creating Gmail Lite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Walking Through the Style Sheet . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Removing Google’s Advertising . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 And Now . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Chapter 5: How Gmail Works . . . . . . . . . . . . . . . . . . . . . . . . 53 What the Devil Is Going On? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Preloading the Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Introducing XMLHttpRequest . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Using XMLHttpRequest Yourself . . . . . . . . . . . . . . . . . . . . . . . 55 Finding XMLHttpRequest within the Gmail code . . . . . . . . . . . . . . 61 Sniffing the Network Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Firing Up Tcpflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Prodding Gmail to Hear It Squeak . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Preparing to Watch the Gmail Boot Sequence . . . . . . . . . . . . . . . . . 67 Cleaning Up the Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Stepping Through the Gmail Boot Sequence . . . . . . . . . . . . . . . . . . . . . 68 Logging In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 The First Cookie . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Loading the Inbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Reading an Individual Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 And Now . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Chapter 6: Gmail and Greasemonkey . . . . . . . . . . . . . . . . . . . . 91 What Is Greasemonkey?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 The Userscripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Displaying Bloglines Within Gmail . . . . . . . . . . . . . . . . . . . . . . 92 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Add a Delete Button. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 GmailSecure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 MailtoComposeInGmail . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Other Userscripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Mark Read Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Multiple Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Hide Invites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Random Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 xii Contents 02_59611x ftoc.qxp 11/28/05 11:05 PM Page xii Chapter 7: Gmail Libraries . . . . . . . . . . . . . . . . . . . . . . . . . 117 PHP — Gmailer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Getting and Installing the Library. . . . . . . . . . . . . . . . . . . . . . . 118 How to Use It . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Perl — Mail::Webmail::Gmail . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Getting and Installing the Library. . . . . . . . . . . . . . . . . . . . . . . 127 Using the Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Python — Libgmail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Getting and Installing the Library. . . . . . . . . . . . . . . . . . . . . . . 131 How to Use It . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Reading the First Message in the Inbox . . . . . . . . . . . . . . . . . . . . 134 Setting Yourselves Up for the Remaining Chapters . . . . . . . . . . . . . . . . . 135 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Chapter 8: Checking for Mail . . . . . . . . . . . . . . . . . . . . . . . . 137 The Basics in Perl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 The Basics in PHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 The Basics in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Building on the Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 New Mail Count in RSS . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 New Mail Count to AOL Instant Messenger . . . . . . . . . . . . . . . . . 144 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Chapter 9: Reading Mail . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Reading Mail with Perl. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 The Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Accessing All the Data of a Message . . . . . . . . . . . . . . . . . . . . . 152 Listing the Mail and Displaying a Chosen Message . . . . . . . . . . . . . 153 Dealing with Attachments. . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Making an RSS Feed of Your Inbox . . . . . . . . . . . . . . . . . . . . . . 155 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Chapter 10: Sending Mail. . . . . . . . . . . . . . . . . . . . . . . . . . 161 Sending Mail with Gmail SMTP . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Sending Mail with Perl . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Part III: Conquering Gmail 167 Chapter 11: Dealing with Labels . . . . . . . . . . . . . . . . . . . . . . 169 Listing the Existing Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Setting New Labels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Creating a New Label . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Removing Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 xiiiContents 02_59611x ftoc.qxp 11/28/05 11:05 PM Page xiii Chapter 12: Addressing Addresses . . . . . . . . . . . . . . . . . . . . . 177 The Contacts List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Importing Contacts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 Showing Your Current Contacts . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Exporting Contacts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Chapter 13: Building an API from the HTML-Only Version of Gmail . . . 183 A First Look at the HTML Version . . . . . . . . . . . . . . . . . . . . . . . . . 183 Introducing Basic Scraping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 HTML::TokeParser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Parsing the Inbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 Retrieving the Individual Page. . . . . . . . . . . . . . . . . . . . . . . . . 192 Dealing with Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Dealing with Other Folders . . . . . . . . . . . . . . . . . . . . . . . . . . 195 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Chapter 14: Exporting Your Mail . . . . . . . . . . . . . . . . . . . . . . 197 Exporting as a Massive Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Converting to Mbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Appending to IMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 And Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Chapter 15: Using Gmail to . . . . . . . . . . . . . . . . . . . . . . . . . 203 Using Gmail as a To-Do List . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Using Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Using gmtodo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Using Gmail to Find Information in RSS Feeds. . . . . . . . . . . . . . . . . . . 205 Using Gmail to Find Torrent Files . . . . . . . . . . . . . . . . . . . . . . 206 Using Gmail as a Notepad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Using Gmail as a Spam Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 An Even Simpler Way of Doing It . . . . . . . . . . . . . . . . . . . . . . 210 Using Gmail as Storage for a Photo Gallery . . . . . . . . . . . . . . . . . . . . . 210 And Now . . . . . .