Linux Socket Programming By Example

• TE AM FL Y Team-Fly® • Page i Linux Socket Programming by Example Warren W. Gay Page ii Linux Socket Programming by Example Copyright © 2000 by Que® All rights reserved. No part of this book shall be reproduced, stored in a retrieval system, or transmitted by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission from the publisher. No patent liability is assumed with respect to the use of the information contained herein. Although every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions. Nor is any liability assumed for damages resulting from the use of the information contained herein. International•Standard•Book•Number:•0-7897-2241-0 Library•of•Congress•Catalog•Card•Number:•99-66454 Printed in the United States of America First Printing: April 2000 02•01•00•4•3•2•1 Trademarks All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Que cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark. Linux® is a registered trademark of Linus Torvalds. Red Hat® LinuxTM is a registered trademark of Red Hat Software. Warning and Disclaimer Every effort has been made to make this book as complete and as accurate as possible, but no warranty or fitness is implied. The information provided is on an "as is" basis. The author and the publisher shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book. Associate Publisher Tracy Dunkelberger Acquisitions Editor Todd Green Development Editor Laura Bulcher Managing Editor Thomas F. Hayes Project Editor Karen S. Shields Copy Editor Victoria Elzey • Indexer Aamir Burki Proofreader Jeanne Clark Technical Editor William Ray Team Coordinator Cindy Teeters Media Developer Jay Payne Interior Designer Karen Ruggles Cover Designer Rader Design Copywriter Eric Borgert Production Lisa England Steve Geiselman Liz Johnston Page iii CONTENTS AT A GLANCE Introduction 1 Part 1: Basic Socket Concepts 5 1 Introducing Sockets 7 2 Domains and Address Families 35 3 Address Conversion Functions 65 4 Socket Types and Protocols 93 5 Binding Addresses to a Socket 115 6 Connectionless-Oriented Protocols 133 7 Connection-Oriented Protocols for Clients 159 8 Connection-Oriented Protocols for Servers 183 9 Hostname and Network Name Lookups 203 Part 2: Advanced Socket Programming 227 10 Using Standard I/O on Sockets 229 11 Concurrent Client Servers 269 12 Socket Options 305 13 329 • Broadcasting with UDP 14 Out-of-Band Data 349 15 Using the inetd Daemon 379 16 Network Security Programming 393 17 Passing Credentials and File Descriptors 427 18 A Practical Network Project 473 Appendixes 505 A Socket Function Quick Reference 507 B Socket-Related Structures Reference 519 C Useful Network Tables 525 Glossary 529 Index 537 Page iv TABLE OF CONTENTS Introduction 1 Part 1 Basic Socket Concepts 5 1 Introducing Sockets 7 A Brief Historical Introduction 8 Understanding Sockets 9 Defining a Socket 9 Using Sockets 10 Referencing Sockets 11 Comparing Sockets to Pipes 12 Creating Sockets 13 Using socketpair(2) in an Example 14 Running the Demonstration Program 16 Performing I/O on Sockets 17 Closing Sockets 21 The shutdown(2) Function 22 Shutting Down Writing to a Socket 22 Dealing with Duplicated Sockets 23 Shutting Down Reading from a Socket 24 Knowing When Not to Use shutdown(2) 24 Writing a Client/Server Example 24 • 2 Domains and Address Families 35 Nameless Sockets 36 Anonymous Calls 36 Generating Addresses 36 Understanding Domains 36 Forming Socket Addresses 37 Examining the Generic Socket Address 37 Forming Local Addresses 38 Forming Traditional Local Addresses 39 Forming Abstract Local Addresses 44 Forming Internet (IPv4) Socket Addresses 47 Understanding Network Byte Order 49 Performing Endian Conversions 50 Page v Initializing a Wild Internet Address 51 Initializing a Specific Internet Address 52 Specifying an X.25 Address 55 Specifying Other Address Families 58 The AF_UNSPEC Address Family 61 3 Address Conversion Functions 65 Internet IP Numbers 66 Internet Address Classes 66 Understanding Netmask Values 67 Allocating IP Addresses 72 Private IP Numbers 72 Reserved IP Numbers 73 Manipulating IP Numbers 73 Using the inet_addr(3) Function 73 The inet_aton(3) Function 77 Using the inet_ntoa(3) Function 80 Using inet_network(3) 83 Using the inet_lnaof(3) Function 85 Using the inet_netof(3) Function 86 Using the inet_makeaddr(3) Function 87 4 Socket Types and Protocols 93 Specifying the Domain of a Socket 94 • Choosing PF_INET or AF_INET 94 Using the PF_LOCAL and AF_LOCAL Macros 95 Using the socket(2) Function 96 Choosing a Socket Type 96 Understanding the SOCK_STREAM Socket Type 97 Understanding the SOCK_DGRAM Socket Type 99 Understanding the SOCK_SEQPACKET Socket Type 100 Choosing a Protocol 101 Using PF_LOCAL and SOCK_STREAM 102 Using PF_LOCAL and SOCK_DGRAM 103 Using PF_INET and SOCK_STREAM 103 Using PF_INET and SOCK_DGRAM 105 Socket Domain and Type Summary 106 Other Linux-Supported Protocols 107 Researching Other Protocols 109 Page vi 5 Binding Addresses to a Socket 115 The Purpose of the bind(2) Function 116 Using the bind(2) Function 116 Obtaining the Socket Address 120 Writing a sock_addr() Function 121 Obtaining a Peer Socket Address 125 Interfaces and Addressing 128 Specifying an Interface Address Example 128 Binding a Specific Interface Address 129 Binding for Any Interface 130 6 Connectionless-Oriented Protocols 133 The Methods of Communication 134 Understanding the Advantages 134 Understanding the Disadvantages of Connectionless Communications 135 Performing Input/Output of Datagrams 136 Introducing the sendto(2) Function 136 Introducing the recvfrom(2) Function 138 Writing a UDP Datagram Server 140 Writing a UDP Datagram Client 146 Testing the Datagram Client and Server 150 Testing with No Server 152 TE AM FL Y Team-Fly® • Testing with Other IP Numbers 153 Leaving out bind(2) in Client Programs 154 Replying to a Wild Address 154 7 Connection-Oriented Protocols for Clients 159 Reviewing the Methods of Communication 160 TCP/IP Handles Lost Packets 160 TCP/IP Handles Duplicated Packets 161 TCP/IP Handles Sequencing 161 TCP/IP Handles Flow Control 161 Understanding the Advantages of TCP/IP 161 Internet Services 162 Examining the /etc/services File 162 Using Function getservent(3) 163 Using the setservent(3) Function 166 Using the endservent(3) Function 167 Page vii Looking up a Service by Name and Protocol 167 Looking up a Service by Port and Protocol 168 Consulting the /etc/protocols File 168 Using the setprotoent(3) Function 171 Using the endprotoent(3) Function 172 Looking up a Protocol by Name 172 Looking up a Protocol by Number 172 Writing a TCP/IP Client Program 173 Introducing the connect(2) Function 173 Preparing to Write the Client Program 174 The daytime Client Program 176 Using connect(2) on SOCK_DGRAM Sockets 180 8 Connection-Oriented Protocols for Servers 183 Understanding the Role of the Server 184 The listen(2) Function 185 Understanding the Connect Queue 186 Specifying a Value for backlog 187 The accept(2) Function Call 188 Understanding the Role of accept(2) 189 Writing a TCP/IP Server 190 Running a Wild Server 196 Modifying the Client Program 197 • 9 Hostname and Network Name Lookups 203 Understanding the Need for Names 204 Using the uname(2) Function 204 Obtaining Hostnames and Domain Names 207 Using Function gethostname 207 Using the getdomainname(2) Function 208 Testing gethostname(2) and getdomainname(2) 208 Resolving Remote Addresses 210 Error Reporting 210 Reporting an h_errno Error 211 Using the gethostbyname(3) Function 212 Applying the gethostbyname(3) Function 214 The gethostbyaddr(3) Function 217 Using the sethostent(3) Function 223 Using the endhostent(3) Function 224 Page viii Part 2 Advanced Socket Programming 227 10 Using Standard I/O on Sockets 229 Understanding the Need for Standard I/O 230 Associating a Socket with a Stream 230 Using fdopen(3) to Associate a Socket with a Stream 231 Closing a Socket Stream 232 Using Separate Read and Write Streams 233 Duplicating a Socket 234 Closing the Dual Streams 234 Winding up Communications 235 Shutting Down the Write Side Only 235 Shutting Down the Read Side Only 237 Shutting Down Both Read and Write Sides 237 Handling Interrupts 238 Handling EINTR for Other Functions 240 Denning Buffer Operation 240 Applying FILE Streams to Sockets 243 Presenting the mkaddr() Function 243 The RPN Calculator Engine Code 249 Trying out the RPN Server 264 11 Concurrent Client Servers 269 • Understanding the Multiple-Client Problem 270 Overview of Server Functions 271 Using fork(2) to Service Multiple Clients 275 Understanding the Overall Server Process 280 Understanding the Child Server Process Flow 281 Understanding Process Termination Processing 282 Designing Servers That Use select(2) 282 Introducing the select(2) Function 283 Manipulating File Descriptor Sets 285 Applying select(2) to a Server 287 Testing the select(2) -Based Server 299 Limitations of the Example 301 12 Socket Options 305 Getting Socket Options 306 Applying getsockopt(2) 307 Page ix Setting Socket Options 310 Applying the setsockopt(2) Function 311 Retrieving the Socket Type• (SO_TYPE) 315 Setting the SO_REUSEADDR Option 317 Setting the SO_LINGER Option 320 Setting the SO_KEEPALIVE Option 323 Setting the SO_BROADCAST Option 324 Setting the SO_OOBINLINE Option 325 Options SO_PASSCRED and SO_PEERCRED 326 13 Broadcasting with UDP 329 Understanding Broadcast Addresses 330 Broadcasting on 255.255.255.255 330 Enhancing the mkaddr.c Subroutine 331 Broadcasting from a Server 332 Receiving Broadcasts 338 Demonstrating the Broadcasts 342 Broadcasting to a Network 343 Starting Broadcasts 343 Receiving Broadcasts 345 Receiving Broadcasts from a Remote Host 345 Troubleshooting Hints 346 14 Out-of-Band Data 349 • Defining Out-of-Band 350 Understanding the Need for Out-of-Band Data 350 Sockets and Out-of-Band Data 351 Variations in Implementation 351 Using Out-of-Band Data 353 Writing Out-of-Band Data 353 Reading Out-of-Band Data 354 Understanding the Signal SIGURG 355 Supporting Subprograms 356 Receiving with the SIGURG Signal 359 Sending Out-of-Band Data 362 Testing the oobrecv and oobsend Programs 365 Page x Understanding the Urgent Pointer 366 Understanding TCP Urgent Mode 366 Urgent Mode When tcp_stdurg=1 369 Receiving Out-of-Band Data Inline 370 Determining the Urgent Pointer 370 Using Out-of-Band Data Inline 371 Limitations of the Urgent Mode Pointer 375 Processing Out-of-Band Data with select(2) 377 15 Using the inetd Daemon 379 Steps Common to Most Servers 380 Introducing inetd 380 The /etc/inetd.conf Configuration File 381 The Design Parameters of inetd Servers 383 Implementing a Simple stream tcp Server 384 Configuring /etc/inetd.conf to Invoke a New Server 385 Disabling the New Service 389 Datagram Servers with inetd 389 Understanding wait and nowait 390 16 Network Security Programming 393 Defining Security 394 The Challenges of Security 394 Identifying Friend or Foe 396 • Securing by Hostname or Domain Name 396 Identifying by IP Number 397 Securing inetd Servers 398 Centralized Network Policy 399 Understanding the TCP Wrapper Concept 399 Determining Access 401 Installing Wrapper and Server Programs 403 Examining Server and Wrapper Logging Code 403 Examining the Datagram Server Code 405 Examining the Simple TCP Wrapper Program 410 Introducing the Client Program 414 Installing and Testing the Wrapper 418 Monitoring the Log Files 419 Starting Your inetd Daemon 419 Page xi Testing the Wrapper Program 420 Testing the Server Timeout 421 Uninstalling the Demonstration Programs 422 Datagram Vulnerability 423 17 Passing Credentials and File Descriptors 427 Problem Statement 428 Introducing Ancillary Data 428 Introducing I/O Vectors 429 The I/O Vector (struct iovec) 429 The readv(2) and writev(2) Functions 430 The sendmsg(2) and recvmsg(2) Functions 432 The sendmsg(2) Function 432 The recvmsg(2) Function 433 Understanding struct msghdr 433 Ancillary Data Structures and Macros 435 Introducing struct cmsghdr Structure 435 Introducing the cmsg(3) Macros 437 Iterating through Ancillary Data 439 Creating Ancillary Data 440 Presenting an Ancillary Data Example 441 The Common Header File common.h 442 The misc.c Module 443 TE AM FL Y Team-Fly® • The recvcred.c Module 443 The Simple Web Server web80 447 The reqport() Function 451 The recv_fd() Function 453 The sockserv Server Program 456 The send_fd() Function 465 Testing the Socket Server 468 Testing sockserv 469 18 A Practical Network Project 473 Problem Statement 474 Solving the Quote Service Problem 474 Obtaining Stock Market Quotes 474 Examining the Quote Server Program 477 Fetching Quotations via get_tickinfo() 484 Page xii Broadcasting Quotes via broadcast() 493 Examining the Client Program 495 Compiling and Running the Demonstration 500 Starting the qserve Quotation Server 501 Starting the mktwatch Client 501 If the finance.yahoo.com Service Changes 503 Appendixes 505 A Socket Function Quick Reference 507 Socket-Specific Functions 507 Socket Addressing 508 Reading of Sockets 508 Writing to Sockets 510 Other Socket I/O 511 Controlling Sockets 512 Network Support Functions 513 Standard I/O Support 515 Hostname Support 515 B Socket-Related Structures Reference 519 Socket Address Structures 519 Miscellaneous Structures 521 I/O-Related Structures 522 C 525 • Useful Network Tables Glossary 529 Index 537 • Page xiii ABOUT THE AUTHOR Warren W. Gay is a supervisor at Mackenzie Financial Corporation in Toronto, Canada. There he supervises a small team of programmers that manage the Mackenzie Investment Management System (IMS). Warren is also the author of Sams Teach Yourself Linux Programming in 24 Hours. Warren has been programming professionally since 1980, using many assembler languages, PL/I, C, and C++. He has been programming for UNIX since 1986 and started programming for Linux in 1994. Linux has allowed him to contribute software packages, such as the ftpbackup program and the rewrite of the popular wavplay program. These and his other Linux packages can be found at sunsite.unc.edu and its mirror ftp sites. Amateur radio is a hobby of Warren's. He holds an advanced amateur radio license and is occasionally active on 75 meters with the radio call sign VE3WWG. Using the 2-meter band on August 3, 1991, he made contact with Musa Manarov, call sign U2MIR, aboard the Soviet MIR space station using a PC and packet radio gear. Warren lives with his wife, Jacqueline, and his three children, Erin, Laura, and Scott, in St. Catharines, Ontario, Canada. • Page xiv DEDICATION This•book•is•dedicated•to•my•loving•wife,•Jackie,•my•daughters,•Erin•and•Laura, and•my•son,•Scott.•Only•through•their•collective•patience•and•support•was•this book•made•possible. ACKNOWLEDGEMENTS First, thanks go to Brian Gill for his enthusiasm and support, which helped to get this project started. Thanks also to Todd Green who took over for Brian as acquisitions editor and to Laura Bulcher as development editor. I also want to thank William Ray for his enthusiasm and effort as the technical editor. Thanks also belong to those at Macmillan USA who expended countless hours doing all of the various jobs that take place in the production of a book. As is so often the case in life, accomplishment is achieved with the effort of many. I would also like to thank the people at Mackenzie Financial Corporation for their support as I juggled my job responsibilities with my writing at home. Particularly, I want to thank Carol Penhale for allowing me to arrange vacation time when I really needed it. I also want to thank Alex Lowitt for his efforts in leasing laptops for Mackenzie employees. The laptop that I used was a great boost to this writing effort. A warm thank-you goes to Darie Urbanky for his assistance in testing a few program examples for me, under Sun's Solaris. To my various other friends, please accept my general thanks for your encouragement and continued support. • Page xv TELL US WHAT YOU THINK! As the reader of this book, you are our most important critic and commentator. We value your opinion and want to know what we're doing right, what we could do better, what areas you'd like to see us publish in, and any other words of wisdom you're willing to pass our way. As an Associate Publisher for Que, I welcome your comments. You can fax, email, or write to let me know what you did or didn't like about this book— as well as what we can do to make our books stronger. Please note that I cannot help you with technical problems related to the topic of this book, and that due to the high volume of mail I receive, I might not be able to reply to every message. When you write, please be sure to include this book's title and author as well as your name and phone or fax number. I will carefully review your comments and share them with the author and editors who worked on the book. Fax: 317-581-4666 Email: quetechnical@macmillanusa.com Mail: Associative Publisher, Programming Que 201 West 103rd Street Indianapolis, IN 46290 USA • Page 1 INTRODUCTION There have been many books written on the topic of computer networking. While many of these are excellent resources for advanced programmers, they tend to be too deep for the beginner who just wants to know how to use it. Why require a potential driver to understand the theory behind his automobile? This book teaches the reader how to use socket programming, as if networking was an appliance that you can turn on and use. Consequently, a ''by example" approach to socket programming is used here. Each chapter builds upon the previous, until all of the basic concepts are mastered in Part 1, "Basic Socket Concepts." Part 2, "Advanced Socket Programming," contains some more advanced topics that might present a challenge for some readers. The last chapter presents a practical application tying together many of the concepts you've learned. The by Example Series How does the by Example series make you a better programmer? The by Example series teaches programming using the best method possible— examples. The text acts as a mentor, looking over your shoulder, providing example programs, and showing you new ways to use the concepts covered in each chapter. While the material is still fresh, you will see example after example, demonstrating ways to use what you just learned. The philosophy of the by Example series is simple: The best way to teach computer programming is with multiple examples. Command descriptions, format syntax, and language references are not enough to teach a newcomer a programming language. Only by taking the components, immediately putting them into use, and running example programs can programming students get more than just a feel for the language. Newcomers who learn only a few basics using examples at every step of the way will automatically know how to write programs using those skills. Who Should Use This Book This book