Advanced Programming in the Unix Environment

W. Richard Stevens, Stephen A. Rago

Mentioned 18

"Stephen Rago's update is a long overdue benefit to the community of professionals using the versatile family of UNIX and UNIX-like operating environments. It removes obsolescence and includes newer developments. It also thoroughly updates the context of all topics, examples, and applications to recent releases of popular implementations of UNIX and UNIX-like environments. And yet, it does all this while retaining the style and taste of the original classic." --Mukesh Kacker, cofounder and former CTO of Pronto Networks, Inc. "One of the essential classics of UNIX programming." --Eric S. Raymond, author of The Art of UNIX Programming "This is the definitive reference book for any serious or professional UNIX systems programmer. Rago has updated and extended the classic Stevens text while keeping true to the original. The APIs are illuminated by clear examples of their use. He also mentions many of the pitfalls to look out for when programming across different UNIX system implementations and points out how to avoid these pitfalls using relevant standards such as POSIX 1003.1, 2004 edition and the Single UNIX Specification, Version 3." --Andrew Josey, Director, Certification, The Open Group, and Chair of the POSIX 1003.1 Working Group "Advanced Programming in the UNIX® Environment, Second Edition, is an essential reference for anyone writing programs for a UNIX system. It's the first book I turn to when I want to understand or re-learn any of the various system interfaces. Stephen Rago has successfully revised this book to incorporate newer operating systems such as GNU/Linux and Apple's OS X while keeping true to the first edition in terms of both readability and usefulness. It will always have a place right next to my computer." --Dr. Benjamin Kuperman, Swarthmore College Praise for the First Edition "Advanced Programming in the UNIX® Environment is a must-have for any serious C programmer who works under UNIX. Its depth, thoroughness, and clarity of explana-tion are unmatched." --UniForum Monthly "Numerous readers recommended Advanced Programming in the UNIX® Environment by W. Richard Stevens (Addison-Wesley), and I'm glad they did; I hadn't even heard of this book, and it's been out since 1992. I just got my hands on a copy, and the first few chapters have been fascinating." --Open Systems Today "A much more readable and detailed treatment of UNIX internals can be found in Advanced Programming in the UNIX® Environment by W. Richard Stevens (Addison-Wesley). This book includes lots of realistic examples, and I find it quite helpful when I have systems programming tasks to do." --RS/Magazine "This is the definitive reference book for any serious or professional UNIX systems programmer. Rago has updated and extended the original Stevens classic while keeping true to the original." --Andrew Josey, Director, Certification, The Open Group, and Chair of the POSIX 1003.1 Working Group For over a decade, serious C programmers have relied on one book for practical, in-depth knowledge of the programming interfaces that drive the UNIX and Linux kernels: W. Richard Stevens' Advanced Programming in the UNIX® Environment . Now, Stevens' colleague Stephen Rago has thoroughly updated this classic to reflect the latest technical advances and add support for today's leading UNIX and Linux platforms. Rago carefully retains the spirit and approach that made this book a classic. Building on Stevens' work, he begins with basic topics such as files, directories, and processes, carefully laying the groundwork for understanding more advanced techniques, such as signal handling and terminal I/O. Substantial new material includes chapters on threads and multithreaded programming, using the socket interface to drive interprocess communication (IPC), and extensive coverage of the interfaces added to the latest version of the POSIX.1 standard. Nearly all examples have been tested on four of today's most widely used UNIX/Linux platforms: FreeBSD 5.2.1; the Linux 2.4.22 kernel; Solaris 9; and Darwin 7.4.0, the FreeBSD/Mach hybrid underlying Apple's Mac OS X 10.3. As in the first edition, you'll learn through example, including more than 10,000 lines of downloadable, ANSI C source code. More than 400 system calls and functions are demonstrated with concise, complete programs that clearly illustrate their usage, arguments, and return values. To tie together what you've learned, the book presents several chapter-length case studies, each fully updated for contemporary environments. Advanced Programming in the UNIX® Environment has helped a generation of programmers write code with exceptional power, performance, and reliability. Now updated for today's UNIX/Linux systems, this second edition will be even more indispensable.

More on

Mentioned in questions and answers.

If you could go back in time and tell yourself to read a specific book at the beginning of your career as a developer, which book would it be?

I expect this list to be varied and to cover a wide range of things.

To search: Use the search box in the upper-right corner. To search the answers of the current question, use inquestion:this. For example:

inquestion:this "Code Complete"

Applying UML and Patterns by Craig Larman.

The title of the book is slightly misleading; it does deal with UML and patterns, but it covers so much more. The subtitle of the book tells you a bit more: An Introduction to Object-Oriented Analysis and Design and Iterative Development.

Masters of doom. As far as motivation and love for your profession go: it won't get any better than what's been described in this book, truthfully inspiring story!

Beginning C# 3.0: An Introduction to Object Oriented Programming

This is the book for those who want to understand the whys and hows of OOP using C# 3.0. You don't want to miss it.

alt text

Mastery: The Keys to Success and Long-Term Fulfillment, by George Leonard

It's about about what mindsets are required to reach mastery in any skill, and why. It's just awesome, and an easy read too.

Pro Spring is a superb introduction to the world of Inversion of Control and Dependency Injection. If you're not aware of these practices and their implications - the balance of topics and technical detail in Pro Spring is excellent. It builds a great case and consequent personal foundation.

Another book I'd suggest would be Robert Martin's Agile Software Development (ASD). Code smells, agile techniques, test driven dev, principles ... a well-written balance of many different programming facets.

More traditional classics would include the infamous GoF Design Patterns, Bertrand Meyer's Object Oriented Software Construction, Booch's Object Oriented Analysis and Design, Scott Meyer's "Effective C++'" series and a lesser known book I enjoyed by Gunderloy, Coder to Developer.

And while books are nice ... don't forget radio!

... let me add one more thing. If you haven't already discovered safari - take a look. It is more addictive than stack overflow :-) I've found that with my google type habits - I need the more expensive subscription so I can look at any book at any time - but I'd recommend the trial to anyone even remotely interested.

(ah yes, a little obj-C today, cocoa tomorrow, patterns? soa? what was that example in that cookbook? What did Steve say in the second edition? Should I buy this book? ... a subscription like this is great if you'd like some continuity and context to what you're googling ...)

Database System Concepts is one of the best books you can read on understanding good database design principles.

alt text

Algorithms in C++ was invaluable to me in learning Big O notation and the ins and outs of the various sort algorithms. This was published before Sedgewick decided he could make more money by dividing it into 5 different books.

C++ FAQs is an amazing book that really shows you what you should and shouldn't be doing in C++. The backward compatibility of C++ leaves a lot of landmines about and this book helps one carefully avoid them while at the same time being a good introduction into OO design and intent.

Here are two I haven't seen mentioned:
I wish I had read "Ruminations on C++" by Koenig and Moo much sooner. That was the book that made OO concepts really click for me.
And I recommend Michael Abrash's "Zen of Code Optimization" for anyone else planning on starting a programming career in the mid 90s.

Perfect Software: And Other Illusions about Testing


Perfect Software: And Other Illusions about Testing by Gerald M. Weinberg

ISBN-10: 0932633692

ISBN-13: 978-0932633699

Rapid Development by McConnell

The most influential programming book for me was Enough Rope to Shoot Yourself in the Foot by Allen Holub.

Cover of the book

O, well, how long ago it was.

I have a few good books that strongly influenced me that I've not seen on this list so far:

The Psychology of Everyday Things by Donald Norman. The general principles of design for other people. This may seem to be mostly good for UI but if you think about it, it has applications almost anywhere there is an interface that someone besides the original developer has to work with; e. g. an API and designing the interface in such a way that other developers form the correct mental model and get appropriate feedback from the API itself.

The Art of Software Testing by Glen Myers. A good, general introduction to testing software; good for programmers to read to help them think like a tester i. e. think of what may go wrong and prepare for it.

By the way, I realize the question was the "Single Most Influential Book" but the discussion seems to have changed to listing good books for developers to read so I hope I can be forgiven for listing two good books rather than just one.

alt text

C++ How to Program It is good for beginner.This is excellent book that full complete with 1500 pages.

Effective C++ and More Effective C++ by Scott Myers.

Inside the C++ object model by Stanley Lippman

I bough this when I was a complete newbie and took me from only knowing that Java existed to a reliable team member in a short time

Not a programming book, but still a very important book every programmer should read:

Orbiting the Giant Hairball by Gordon MacKenzie

The Pragmatic programmer was pretty good. However one that really made an impact when I was starting out was :

Windows 95 System Programming Secrets"

I know - it sounds and looks a bit cheesy on the outside and has probably dated a bit - but this was an awesome explanation of the internals of Win95 based on the Authors (Matt Pietrek) investigations using his own own tools - the code for which came with the book. Bear in mind this was before the whole open source thing and Microsoft was still pretty cagey about releasing documentation of internals - let alone source. There was some quote in there like "If you are working through some problem and hit some sticking point then you need to stop and really look deeply into that piece and really understand how it works". I've found this to be pretty good advice - particularly these days when you often have the source for a library and can go take a look. Its also inspired me to enjoy diving into the internals of how systems work, something that has proven invaluable over the course of my career.

Oh and I'd also throw in effective .net - great internals explanation of .Net from Don Box.

I recently read Dreaming in Code and found it to be an interesting read. Perhaps more so since the day I started reading it Chandler 1.0 was released. Reading about the growing pains and mistakes of a project team of talented people trying to "change the world" gives you a lot to learn from. Also Scott brings up a lot of programmer lore and wisdom in between that's just an entertaining read.

Beautiful Code had one or two things that made me think differently, particularly the chapter on top down operator precedence.


@Juan: I know Juan, I know - but there are some things that can only be learned by actually getting down to the task at hand. Speaking in abstract ideals all day simply makes you into an academic. It's in the application of the abstract that we truly grok the reason for their existence. :P

@Keith: Great mention of "The Inmates are Running the Asylum" by Alan Cooper - an eye opener for certain, any developer that has worked with me since I read that book has heard me mention the ideas it espouses. +1

I found the The Algorithm Design Manual to be a very beneficial read. I also highly recommend Programming Pearls.

This one isnt really a book for the beginning programmer, but if you're looking for SOA design books, then SOA in Practice: The Art of Distributed System Design is for you.

For me it was Design Patterns Explained it provided an 'Oh that's how it works' moment for me in regards to design patterns and has been very useful when teaching design patterns to others.

Code Craft by Pete Goodliffe is a good read!

Code Craft

The first book that made a real impact on me was Mastering Turbo Assembler by Tom Swan.

Other books that have had an impact was Just For Fun by Linus Torvalds and David Diamond and of course The Pragmatic Programmer by Andrew Hunt and David Thomas.

In addition to other people's suggestions, I'd recommend either acquiring a copy of SICP, or reading it online. It's one of the few books that I've read that I feel greatly increased my skill in designing software, particularly in creating good abstraction layers.

A book that is not directly related to programming, but is also a good read for programmers (IMO) is Concrete Mathematics. Most, if not all of the topics in it are useful for programmers to know about, and it does a better job of explaining things than any other math book I've read to date.

For me "Memory as a programming concept in C and C++" really opened my eyes to how memory management really works. If you're a C or C++ developer I consider it a must read. You will defiantly learn something or remember things you might have forgotten along the way.

Agile Software Development with Scrum by Ken Schwaber and Mike Beedle.

I used this book as the starting point to understanding Agile development.

Systemantics: How Systems Work and Especially How They Fail. Get it used cheap. But you might not get the humor until you've worked on a few failed projects.

The beauty of the book is the copyright year.

Probably the most profound takeaway "law" presented in the book:

The Fundamental Failure-Mode Theorem (F.F.T.): Complex systems usually operate in failure mode.

The idea being that there are failing parts in any given piece of software that are masked by failures in other parts or by validations in other parts. See a real-world example at the Therac-25 radiation machine, whose software flaws were masked by hardware failsafes. When the hardware failsafes were removed, the software race condition that had gone undetected all those years resulted in the machine killing 3 people.

It seems most people have already touched on the some very good books. One which really helped me out was Effective C#: 50 Ways to Improve your C#. I'd be remiss if I didn't mention The Tao of Pooh. Philosophy books can be good for the soul, and the code.

Discrete Mathematics For Computer Scientists

Discrete Mathematics For Computer Scientists by J.K. Truss.

While this doesn't teach you programming, it teaches you fundamental mathematics that every programmer should know. You may remember this stuff from university, but really, doing predicate logic will improve you programming skills, you need to learn Set Theory if you want to program using collections.

There really is a lot of interesting information in here that can get you thinking about problems in different ways. It's handy to have, just to pick up once in a while to learn something new.

I saw a review of Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools on a blog talking also about XI-Factory, I read it and I must say this book is a must read. Altough not specifically targetted to programmers, it explains very clearly what is happening in the programming world right now with Model-Driven Architecture and so on..

Solid Code Optimizing the Software Development Life Cycle

Although the book is only 300 pages and favors Microsoft technologies it still offers some good language agnostic tidbits.

Managing Gigabytes is an instant classic for thinking about the heavy lifting of information.

My vote is "How to Think Like a Computer Scientist: Learning With Python" It's available both as a book and as a free e-book.

It really helped me to understand the basics of not just Python but programming in general. Although it uses Python to demonstrate concepts, they apply to most, if not all, programming languages. Also: IT'S FREE!

Object-Oriented Programming in Turbo C++. Not super popular, but it was the one that got me started, and was the first book that really helped me grok what an object was. Read this one waaaay back in high school. It sort of brings a tear to my eye...

My high school math teacher lent me a copy of Are Your Lights Figure Problem that I have re-read many times. It has been invaluable, as a developer, and in life generally.

I'm reading now Agile Software Development, Principles, Patterns and Practices. For those interested in XP and Object-Oriented Design, this is a classic reading.

alt text

Kernighan & Plauger's Elements of Programming Style. It illustrates the difference between gimmicky-clever and elegant-clever.

to get advanced in prolog i like these two books:

The Art of Prolog

The Craft of Prolog

really opens the mind for logic programming and recursion schemes.

Here's an excellent book that is not as widely applauded, but is full of deep insight: Agile Software Development: The Cooperative Game, by Alistair Cockburn.

What's so special about it? Well, clearly everyone has heard the term "Agile", and it seems most are believers these days. Whether you believe or not, though, there are some deep principles behind why the Agile movement exists. This book uncovers and articulates these principles in a precise, scientific way. Some of the principles are (btw, these are my words, not Alistair's):

  1. The hardest thing about team software development is getting everyone's brains to have the same understanding. We are building huge, elaborate, complex systems which are invisible in the tangible world. The better you are at getting more peoples' brains to share deeper understanding, the more effective your team will be at software development. This is the underlying reason that pair programming makes sense. Most people dismiss it (and I did too initially), but with this principle in mind I highly recommend that you give it another shot. You wind up with TWO people who deeply understand the subsystem you just built ... there aren't many other ways to get such a deep information transfer so quickly. It is like a Vulcan mind meld.
  2. You don't always need words to communicate deep understanding quickly. And a corollary: too many words, and you exceed the listener/reader's capacity, meaning the understanding transfer you're attempting does not happen. Consider that children learn how to speak language by being "immersed" and "absorbing". Not just language either ... he gives the example of some kids playing with trains on the floor. Along comes another kid who has never even SEEN a train before ... but by watching the other kids, he picks up the gist of the game and plays right along. This happens all the time between humans. This along with the corollary about too many words helps you see how misguided it was in the old "waterfall" days to try to write 700 page detailed requirements specifications.

There is so much more in there too. I'll shut up now, but I HIGHLY recommend this book!

alt text

The Back of the Napkin, by Dan Roam.

The Back of the Napkin

A great book about visual thinking techniques. There is also an expanded edition now. I can't speak to that version, as I do not own it; yet.

Agile Software Development by Alistair Cockburn

Do users ever touch your code? If you're not doing solely back-end work, I recommend About Face: The Essentials of User Interface Design — now in its third edition (linked). I used to think my users were stupid because they didn't "get" my interfaces. I was, of course, wrong. About Face turned me around.

"Writing Solid Code: Microsoft's Techniques for Developing Bug-Free C Programs (Microsoft Programming Series)" by Steve MacGuire.

Interesting what a large proportion the books mentioned here are C/C++ books.

While not strictly a software development book, I would highly recommend that Don't Make me Think! be considered in this list.

As so many people have listed Head First Design Patterns, which I agree is a very good book, I would like to see if so many people aware of a title called Design Patterns Explained: A New Perspective on Object-Oriented Design.

This title deals with design patterns excellently. The first half of the book is very accessible and the remaining chapters require only a firm grasp of the content already covered The reason I feel the second half of the book is less accessible is that it covers patterns that I, as a young developer admittedly lacking in experience, have not used much.

This title also introduces the concept behind design patterns, covering Christopher Alexander's initial work in architecture to the GoF first implementing documenting patterns in SmallTalk.

I think that anyone who enjoyed Head First Design Patterns but still finds the GoF very dry, should look into Design Patterns Explained as a much more readable (although not quite as comprehensive) alternative.

Even though i've never programmed a game this book helped me understand a lot of things in a fun way.

How influential a book is often depends on the reader and where they were in their career when they read the book. I have to give a shout-out to Head First Design Patterns. Great book and the very creative way it's written should be used as an example for other tech book writers. I.e. it's written in order to facilitate learning and internalizing the concepts.

Head First Design Patterns

97 Things Every Programmer Should Know

alt text

This book pools together the collective experiences of some of the world's best programmers. It is a must read.

Extreme Programming Explained: Embrace Change by Kent Beck. While I don't advocate a hardcore XP-or-the-highway take on software development, I wish I had been introduced to the principles in this book much earlier in my career. Unit testing, refactoring, simplicity, continuous integration, cost/time/quality/scope - these changed the way I looked at development. Before Agile, it was all about the debugger and fear of change requests. After Agile, those demons did not loom as large.

One of my personal favorites is Hacker's Delight, because it was as much fun to read as it was educational.

I hope the second edition will be released soon!

You.Next(): Move Your Software Development Career to the Leadership Track ~ Michael C. Finley (Author), Honza Fedák (Author) link text

I've been arounda while, so most books that I have found influential don't necessarily apply today. I do believe it is universally important to understand the platform that you are developing for (both hardware and OS). I also think it's important to learn from other peoples mistakes. So two books I would recommend are:

Computing Calamities and In Search of Stupidity: Over Twenty Years of High Tech Marketing Disasters

Working Effectively with Legacy Code is a really amazing book that goes into great detail about how to properly unit test your code and what the true benefit of it is. It really opened my eyes.

I'm trying to create a daemon in python. I've found the following question, which has some good resources in it which I am currently following, but I'm curious as to why a double fork is necessary. I've scratched around google and found plenty of resources declaring that one is necessary, but not why.

Some mention that it is to prevent the daemon from acquiring a controlling terminal. How would it do this without the second fork? What are the repercussions?

Looking at the code referenced in the question, the justification is:

# Fork a second child and exit immediately to prevent zombies.  This
# causes the second child process to be orphaned, making the init
# process responsible for its cleanup.  And, since the first child is
# a session leader without a controlling terminal, it's possible for
# it to acquire one by opening a terminal in the future (System V-
# based systems).  This second fork guarantees that the child is no
# longer a session leader, preventing the daemon from ever acquiring
# a controlling terminal.

So it is to ensure that the daemon is re-parented onto init (just in case the process kicking off the daemon is long lived), and removes any chance of the daemon reacquiring a controlling tty. So if neither of these cases apply, then one fork should be sufficient. "Unix Network Programming - Stevens" has a good section on this.

Are there any good books for a relatively new but not totally new *nix user to get a bit more in depth knowledge (so no "Linux for dummies")? For the most part, I'm not looking for something to read through from start to finish. Rather, I'd rather have something that I can pick up and read in chunks when I need to know how to do something or whenever I have one of those "how do I do that again?" moments. Some areas that I'd like to see are:

  • command line administration
  • bash scripting
  • programming (although I'd like something that isn't just relevant for C programmers)

I'd like this to be as platform-independent as possible (meaning it has info that's relevant for any linux distro as well as BSD, Solaris, OS X, etc), but the unix systems that I use the most are OS X and Debian/Ubuntu. So if I would benefit the most from having a more platform-dependent book, those are the platforms to target.

If I can get all this in one book, great, but I'd rather have a bit more in-depth material than coverage of everything. So if there are any books that cover just one of these areas, post it. Hell, post it even if it's not relevant to any of those areas and you think it's something that a person in my position should know about.

I've wiki'd this post - could those with sufficient rep add in items to it.

System administration, general usage books


Specific tools (e.g. Sendmail)

Various of the books from O'Reilly and other publishers cover specific topics. Some of the key ones are:

Some of these books have been in print for quite a while and are still relevant. Consequently they are also often available secondhand at much less than list price. Amazon marketplace is a good place to look for such items. It's quite a good way to do a shotgun approach to topics like this for not much money.

As an example, in New Zealand technical books are usurously expensive due to a weak kiwi peso (as the $NZ is affectionately known in expat circles) and a tortuously long supply chain. You could spend 20% of a week's after-tax pay for a starting graduate on a single book. When I was living there just out of university I used this type of market a lot, often buying books for 1/4 of their list price - including the cost of shipping to New Zealand. If you're not living in a location with tier-1 incomes I recommend this.

E-Books and on-line resources (thanks to israkir for reminding me):

  • The Linux Documentation project (, has many specific topic guides known as HowTos that also often concern third party OSS tools and will be relevant to other Unix variants. It also has a series of FAQ's and guides.

  • Unix Guru's Universe is a collection of unix resources with a somewhat more old-school flavour.

  • Google. There are many, many unix and linux resources on the web. Search strings like unix commands or learn unix will turn up any amount of online resources.

  • Safari. This is a subscription service, but you can search the texts of quite a large number of books. I can recommend this as I've used it. They also do site licences for corporate customers.

Some of the philosophy of Unix:

I recommend the Armadillo book from O'Reilly for command line administration and shell scripting.

alt text


Unix Programming Environment by Kernighan and Pike will give you solid foundations on all things Unix and should cover most of your questions regarding shell command line scripting etc.

The Armadillo book by O'Reilly will add the administration angle. It has served me well!

Good luck!

The aforementioned Unix Power Tools is a must. Other classics are sed&awk and Mastering Regular Expressions. I also like some books from the O'Reilly "Cookbook" series:

I'm a beginning C++ programmer / network admin, but I figure I can learn how to do this if someone points me in the right direction. Most of the tutorials are demonstrated using old code that no longer works for some reason.

Since I'm on Linux, all I need is an explanation on how to write raw Berkeley sockets. Can someone give me a quick run down?

Read Unix Network Programming by Richard Stevens. It's a must. It explains how it all works, gives you code, and even gives you helper methods. You might want to check out some of his other books. Advanced Programming In The Unix Enviernment is a must for lower level programming in Unix is general. I don't even do stuff on the Unix stack anymore, and the stuf from these books still helps how I code.

For TCP client side:

Use gethostbyname to lookup dns name to IP, it will return a hostent structure. Let's call this returned value host.

hostent *host = gethostbyname(HOSTNAME_CSTR);

Fill the socket address structure:

sockaddr_in sock;
sock.sin_family = AF_INET;
sock.sin_port = htons(REMOTE_PORT);
sock.sin_addr.s_addr = ((struct in_addr *)(host->h_addr))->s_addr;

Create a socket and call connect:

s = socket(AF_INET, SOCK_STREAM, 0); 
connect(s, (struct sockaddr *)&sock, sizeof(sock))

For TCP server side:

Setup a socket

Bind your address to that socket using bind.

Start listening on that socket with listen

Call accept to get a connected client. <-- at this point you spawn a new thread to handle the connection while you make another call to accept to get the next connected client.

General communication:

Use send and recv to read and write between the client and server.

Source code example of BSD sockets:

You can find some good example code of this at wikipedia.

Further reading:

I highly recommend this book and this online tutorial:

alt text


I know execvp can be used to execute simple commands as follows:

char* arg[] = {"ls", "-l", NULL};

I want to know what goes on in here when I run execvp. In man page it says execvp replaces the image of the process image with the new one. However here I am running a command not an executable.

To be specific, say there is a command that specifically requires an input e.g. cat. If I have a text file text.txt which contains the file name expected for cat and I redirect stdin to the file stream of the file, would the output of execle("cat","cat",NULL) or execvp("cat", arg) (obviously where arg stores "cat" and NULL) result in the output in the console as the cat /filename would? My intuition is I have to read the file and may be parse it to store the arguments in the arg. However I want to make sure.

Thanks in advance!

Regarding your questions:

In man page it says execvp replaces the image of the process image with the new one. However here I am running a command not an executable.

Long-long time ago shell was very limited and almost all UNIX commands was standalone executables. Now, mostly for speed purposes some subset of UNIX commands is implemented inside shell itself, those commands are called builtins. You can check whatever command is implemented in your shell as built-in or not via type command:

λ ~/ type echo
echo is a shell builtin

(Full list of builtins with descriptions can be found in man pages to your shell e.g. man bash-builtins or man builtin.)

But still most of the commands still have their executable-counterpart:

λ ~/ whereis echo

So in your specific case when you are running:

char* arg[] = {"ls", "-l", NULL};

You are actually replacing address space of current process with address space of (most likely) /bin/ls.

My intuition is I have to read the file and may be parse it to store the arguments in the arg.

Indeed you you have. But you also may use some in-kernel functions for that aka "shebang":
Instead of putting file name in separate file add so-called shebang as the first line of the file you want to cat:


And add chmod +x to it. Then you can run it as executable (via any of exec functions or shell):

λ ~/tmp/ printf '#!/bin/cat\nTEST\n' > cat_me
λ ~/tmp/ chmod +x cat_me
λ ~/tmp/ ./cat_me 

Of cause it's has a drawback of printing shebang itself with file but still it's fun to do it in-kernel =)

BTW. Problem that you described if so common that there is a special executable called xargs which (in very simplified explanation) executes given program on list of arguments passed via stdin. For more information consult with man xargs.

For easy memorization of exec-family I often use following table:

           Figure 8.14. Differences among the six exec functions
| Function | pathname | filename | agr list | argv[] | environ | envp[] |
|  execl   |    *     |          |     *    |        |    *    |        |
|  execlp  |          |    *     |     *    |        |    *    |        |
|  execle  |    *     |          |     *    |        |         |   *    |
|  execv   |    *     |          |          |    *   |    *    |        |
|  execvp  |          |    *     |          |    *   |    *    |        |
|  execve  |    *     |          |          |    *   |         |   *    |
|  letter  |          |    p     |     l    |    v   |         |   e    |

So in your case execvp takes filename, argv(v) and environ(e). Then it's tries to "guess" pathname (aka full path) by appending filename (in your case cat) to each path component in PATH until it find path with executable filename.

Much more information about whats going on under the exec's hood (including inheritance stuff) can be found in Advanced Programming in the UNIX Environment (2nd Edition) by W. Richard Stevens and Stephen A. Rago aka APUE2.
If you are interested in UNIX internals you should probably read it.

Hi I'd like to read good tutorials on setjmp/longjmp in C. It'd be better if there're examples which are real rather than artificial.


Then you should read Advanced Programming in the UNIX(R) Environment (2nd Edition) Here's the sample you're looking for (just a preview)

I was just thrust into Linux programming (Red Hat) after several years of C++ on Win32. So I am not looking for the basics of programming. Rather I am looking to get up to speed with things unique to the Linux programming world, such as packages, etc. In other words, I need to know everything in without spending 3K. Any ideas of how I can acquire that knowledge quickly (and relatively cheaply)?

Update: The things that I am used to doing on Windows like building .exe and dlls using VC++, creating install scripts etc are just done differently on Linux. They use things like yum, make and make install, etc. Things like dependency walker that I take for granted in the windows world constantly send me to google while doing linux. Is there a 'set' of new skills somewhere that I can browse or is this more of a learn as you go?

The primary problem is this: As a very experienced programmer in Windows,I am having to ask simple questions like what's the difference between usr\bin and usr\local\bin and I would like to be prepared.

For POSIX and such I can recommend Advanced Programming in the UNIX Environment and having a bookmark to The single UNIX Specification.

For GCC/GDB and those tools I'm afraid I can't give you any good recommendation.

Hope that helps anyway.

Edit: Duck was slightly faster.

Edited because I had to leave a meeting when I originally submitted this, but wanted to complete the information

Half of that material is learning about development in a Unix-like environment, and for that, I'd recommend a book since it's tougher to filter out useful information from the start.

I'd urge you to go to a bookstore and browse through these books:

  • Advanced Programming in the Unix Environment by Stevens and Rago - this book covers threads, networking, IPC, signals, files, process management
  • Unix Network Programming, Volume 1 by Stevens - This book is focused on network programming techniques, design - you might not need this until much later
  • Unix/Linux System Administration - This book covers the more system administrator side of stuff, like directory structure of most Unix and Linux file systems (Linux distributions are more diverse than their Unix-named counterparts in how they might structure their file system)

    Other information accessible online:

  • GCC Online Manual - the comprehensive GNU GCC documentation

  • Beej's network programming guide - A really well written tutorial to network programming with the use of the BSD API. If you have done work with winsock, this should be mostly familiar to you.
  • Red Hat Enterprise Linux 5's Deployment Guide - talks specifically about Red Hat EL 5's basic administrative/deployment, like installing with package manager, a Red Hat system's directory structure...
  • make - Wikipedia article that will have links to the various make documentation out there
  • binutils - These are the Linux tools used for manipulating object/binaries.
  • GNU Build System - Wikipedia article about the traditional build system of GNU software, using autoconf/automake/autogen

Additionally, you will want to learn about ldd, which is like dependency walker in Windows. It lists a target binary's dependencies, if it has any.

And for Debugging, check out this StackOverflow thread which talks about a well written GDB tutorial and also links to an IBM guide.

Happy reading.

I am writing a simple SMTP server and client. Server is in two parts receiver-SMTP and Sender SMTP. both will run in two different daemon services. The two modes this can run in is 'remote' and 'local'.

Since i am new to network programming I am having difficulty in even getting started. Any help in form of text, sample or skeleton code of an SMTP will help me a lot in clearing my doubts,

If your primary motive is to learn network programming and writing daemons, then I would recommend reading Beej's Guide to Network Programming and Advanced Programming in the Unix Environment. These don't provide straight up SMTP implementations but will give a good foundation to implement any protocol.

I'm learning to work with fork(), and I have some questions.

Consider the following code:

#include <stdio.h>
#include <unistd.h>

int main()
   int i;

   for(i = 0; i < 5; i++)
      printf("%d", i);


When I output to a terminal, I get the result I expect (i.e.: 0,1,1,1,2,2,2,...). But when I output to a file, the result is completely different:

  • Case 1: (output to terminal, e.g.: ./a.out):

    Result is: 0,1,1,1,2,2,2,...

  • Case 2: (output to file, e.g.: ./a.out > output_file)

    Result is: 0,1,2,3,4,0,1,2,3,4,0,1,2,3,4,...

Why it is like this?

Why do I have different output between a terminal and a file when forking?

C standard library functions use internal buffering for speed up. Most implementations use fully buffered IO for file streams, line buffered for stdin/stdout and unbuffered for stderr.

So your problem can be solved in number of ways:

  1. Use explicit buffer flush before fork via fflush(3)
  2. Set buffer type manually via setvbuf(3)
  3. Use write(2) instead of stdlib's printf(3)
  4. Output to stderr by default via fprintf(3) *
  5. Exit with _exit(2) in forked processes instead of exit(3) **

Last two may not work as expected if:
* your implementation does not use unbuffered writes to stderr by default (Which is required by ISO C)
** you have written more than default buffer size in child and if was automatically flushed.

PS. Yet again, if you need deeper knowledge of standard library functions and buffering I recommend reading Advanced Programming in the UNIX Environment (2nd Edition) by W. Richard Stevens and Stephen A. Rago.

PPS. btw, your question is a very popular interview question for C/C++ programmer position.

It is possible to do IPC (inter process communication) using signal catch and signal raise?

I made two programs. In the first program I did handling of signals, and in the other program I just raised signal which I want to handle in another program. I'ts working fine for me but I want to do communication between these two programs using signals and also want to send some bytes of data with this raise signal. How can I do this?

I want to pass messages with this signal also. Can i do it? It is possible?

And also, what are the disadvantages and advantages of IPC mechanisms using signals?

The following is working code of my two programs. Ising this, I am able to just raise signals and catch signals, but I want to pass data from one program to another.

In the second program, I used the first program's process ID. How can I make it dynamic.?

first program :

/* Example of using sigaction() to setup a signal handler with 3 arguments
 * including siginfo_t.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <string.h>

static void hdl (int sig, siginfo_t *siginfo, void *context)
    printf("sig no = %d \n", sig);
    if(sig == SIGINT)
    printf ("Sending PID: %ld, UID: %ld\n",
            (long)siginfo->si_pid, (long)siginfo->si_uid);

int main (int argc, char *argv[])
    struct sigaction act;


    act.sa_sigaction = &hdl;
    act.sa_flags = SA_SIGINFO;

    if (sigaction(SIGUSR1, &act, NULL) < 0) {
        perror ("sigaction SIGUSR1");
        return 1;
    if (sigaction(SIGINT, &act, NULL) < 0) {
        perror ("sigaction SIGINT");
        return 1;

    while (1)

    return 0;

second program

#include  <stdio.h>
#include  <signal.h>

void  main(void)

   while (1)
        kill(11558, SIGUSR1);


It is possible to do IPC (inter process communication) using signal catch and signal raise?

Yes and no. Considering signals only, you can send a signal to another process, but you can't send anything other than just a signal.

I want to pass messages with this signal also. Can i do it? It is possible?

No, not the way you're trying to. You can use sockets, files, pipes, or named pipes to do this. If you want to learn more about UNIX IPC, read Advanced Programming in the UNIX Environment.

I've been trying to figure out how TTY drivers works for a while now (with full understanding each kernel's implementation may be different), and came across a nice article: The TTY demystified

I notice however that it claims xterm does not have a stdin, stdout or stderr. How does xterm and other terminal emulators get the input from bash and other child processes so it can print to the window? Does the terminal emulator have a connection to a TTY driver at all?

I don't believe the claim that xterm(1) has no stdin, stdout, or stderr makes much sense. And, just skimming that article, I think they were left blank on the diagram because they didn't figure much into what he was describing.

You can check your own terminals' file descriptors easily enough. The urxvt(1) terminals that I normally use look like this:

lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 0 -> /home/sarnold/.xsession-errors
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 1 -> /home/sarnold/.xsession-errors
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 2 -> /home/sarnold/.xsession-errors

An xterm(1) I started from one of those urxvt(1) terminals looks like this:

lrwx------ 1 sarnold sarnold 64 2011-11-15 16:45 0 -> /dev/pts/1
lrwx------ 1 sarnold sarnold 64 2011-11-15 16:45 1 -> /dev/pts/1
lrwx------ 1 sarnold sarnold 64 2011-11-15 16:45 2 -> /dev/pts/1

And an xterm(1) I started using my window manager's dmenu(1) launcher looks like this:

lrwx------ 1 sarnold sarnold 64 2011-11-15 16:46 0 -> /dev/null
lrwx------ 1 sarnold sarnold 64 2011-11-15 16:46 1 -> /home/sarnold/.xsession-errors
lrwx------ 1 sarnold sarnold 64 2011-11-15 16:46 2 -> /home/sarnold/.xsession-errors

The important part to remember about stdin, stdout, and stderr is that they are completely unrelated to the graphical display that the terminal emulators offer. If your terminal program needs to write an error to stderr, say via perror(3), its output might go to the terminal where you started it or to a session error log such as ~/.xsession-errors. If you log in via ssh(1) and start your xterm(1) like this:

DISPLAY=:1 xterm -e 'echo hello ; sleep 10`

You'll see it start up and display hello. (Assuming your DISPLAY matches mine.) If you change the command to:

DISPLAY=:1 xterm -fn fiddly -e 'echo hello ; sleep 10'

You'll see that the error message about the incorrect font is sent to the terminal where you started xterm(1) -- not to its own graphical interface. (This is a bit funny, because if you change SHELL to something that doesn't exist, or attempt to execute something that doesn't exist, the error message will be printed within the graphical window instead of to the standard error.)

A file that is also open in my terminal emulators is ptmx(4):

lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 0 -> /home/sarnold/.xsession-errors
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 1 -> /home/sarnold/.xsession-errors
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 2 -> /home/sarnold/.xsession-errors
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 3 -> anon_inode:[eventpoll]
lr-x------ 1 sarnold sarnold 64 2011-11-10 21:33 4 -> pipe:[16398]
l-wx------ 1 sarnold sarnold 64 2011-11-10 21:33 5 -> pipe:[16398]
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 6 -> socket:[16399]
lrwx------ 1 sarnold sarnold 64 2011-11-10 21:33 7 -> /dev/ptmx

Opening the ptmx(4) device gives the terminal emulator the pipe connections for the PTY master and slave. It is to these pipes that the terminal emulator software communicates with the clients and the kernel.

I can't do the tty layer justice; I strongly recommend reading the pty section from Advanced Programming in the Unix Environment, 2nd edition for full details. The book (and source code include writing your own pty driver which can be used to implement script(1)-like functionality or tricking the standard C IO streams into using line-buffering rather than block-buffering when the program doesn't provide any command line options to control this natively.

I’m a C# and Java serverside windows programmer by day and want to learn C++ on Unix in my spare time.

I have very limited knowledge of C++ (from my university days).

I currently own a dell laptop running Vista. Should I create a dual boot system or buy a new machine?

Which Unix OS should I go for?

Which IDE should I use? …. NetBeans looks ok?

Which books should I read? I’m interested in texts covering the fundamentals, libraries, network programming, distributed systems.


Books advice:

IDE: my favorite choice would be Eclipse, which has a c++ plugin (and many more interesting plugins), but I would also advice you to learn to use at least one among the Sacred Monsters: VI/VIM and Emacs.

OS: I would go for a dual boot system with some linux distro, maybe Ubuntu.

I think you could go for Dual boot as klez pointed out.

I have a HP2000z dual boot with XP and Ubuntu, and i really like Ubuntu a lot!

For C++, you could give Code::Blocks a try. Its open source.

Adding to orsogufo, I also suggest Unix Network programming Vol I and Vol II by Richard Stevens for Network related topics.

Where is a good place to start if one is interested in Unix systems programming?

Any recommended reading, tutorials etc that are aimed at the beginner?

What knowledge is needed to start with systems programming?

Start with Mark Rochkind's "Advanced Unix Programming" if you can find it. Then graduate to Stevens "Advanced Programming in the Unix Environment".

I discovered this too for anyone interested. Apparently it is the "New standard" for linux programming. alt text

The Linux Programming Interface: A Linux and UNIX System Programming Handbook

Stevens is the bible. Read and understand this and his other books and you have most of what you need.

I am currently trying to understand how to use sockets to make a client/server program in C. I have been reading various tutorials around the Internet in hopes to try and make a small echo server that can deal with multiple clients at the same time. Whenever the client sends the server a message, the server is supposed to echo is back to the client. The code that I created is a combination of both a tutorial from a lecture at school (which explained how to create the client and server) as well as an example I found on here on stackoverflow (which showed how to make the functions to actually echo the messages). I am hoping that someone can explain to me what I am missing to make this program work correctly. Here is the client code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>

char buf[80];
struct sockaddr myname;

void replyBack(FILE *fp, int sockfd) {
char sendline[1000], recvline[1000];
printf("Enter your echo: \n");
while(fgets(sendline,1000,stdin) != NULL) {
    if(read(sockfd,recvline,1000) == 0) {
        printf("str_cli: server terminated prematurely");
    fputs(recvline, stdout);

main() {
int sock, adrlen, cnt;

sock = socket(AF_UNIX, SOCK_STREAM, 0);
if(sock < 0) {
    printf("client socket failure%d\n", errno);
    printf("client: ");

myname.sa_family = AF_UNIX;
strcpy(myname.sa_data, "/tmp/billb");
adrlen = strlen(myname.sa_data) + sizeof(myname.sa_family);

if(connect(sock, &myname, adrlen) < 0) {
    printf("client connect failure %d\n", errno);
    perror("client: ");


And here is the server code:

#include <stdio.h>
#include <errno.h>
#include <signal.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>

struct sockaddr myname;
char buf[80];

void echo(int sockfd) {
ssize_t n;
int write_err;
char buf[1000];
char * send_start_pos;
while(1) {
    bytes_in = read(sockfd, buf, 1000);
    if(bytes_in < 1) {
        if(errno == EINTR)
    bytes_remaining = bytes_in;
    send_start_pos = buf;
    write_err = 0;

    while((bytes_remaining > 0) && !(write_err)) {
        bytes_out = write(sockfd, send_start_pos,           
        if(bytes_out < 0) {
            if(errno == EINTR)
            write_err = 1;
    bytes_remaining -= bytes_out;
    send_start_pos += bytes_out;

main() {
int sock, new_sd, adrlen, cnt;

sock = socket(AF_UNIX, SOCK_STREAM, 0);
if(sock < 0) {
    printf("server socket failure %d\n", errno);
    perror("server: ");

myname.sa_family = AF_UNIX;
strcpy(myname.sa_data, "/tmp/billb");
adrlen = strlen(myname.sa_data) + sizeof(myname.sa_family);

unlink("/tmp/billb"); /*defensive programming */
if(bind(sock, &myname, adrlen) < 0) {
    printf("server bind failure%d\n", errno);
    perror("server: ");

if(listen(sock, 5) < 0) {
    printf("server listen failure %d\n", errno);
    perror("server: ");

while(1) {
    if(new_sd = accept(sock, &myname, &adrlen) < 0) {
        printf("server accept failure %d\n", errno);
        perror("server: ");

    printf("Socket address in server %d is %s, %s\n", 
        getpid(), myname.sa_data, myname.sa_data);

    if(fork() == 0) {

The problem when I run the program is that the client is accepting the message but then does not actually send it to the server, so the server can never echo it back.

I know this is probably basic stuff so I appreciate your patience and time!

adrlen = strlen(myname.sa_data) + sizeof(myname.sa_family);

This should be sizeof myname. The unix(7) sockets are actually defined to have a sockaddr_un like this:

       #define UNIX_PATH_MAX    108

       struct sockaddr_un {
           sa_family_t sun_family;               /* AF_UNIX */
           char        sun_path[UNIX_PATH_MAX];  /* pathname */

You've given it 10+4 as the size. Which should almost work -- it doesn't count the terminating NUL byte at the end of the filename -- but even with the + 1 added, I'd feel better if you're passing in exactly the size of object you have. (Be careful of strlen(3) on strings. Almost always you need a + 1 in any expression that involves strlen(3). Funny.)

while(fgets(sendline,1000,stdin) != NULL) {

And here you're sending all 1000 bytes of your buffer, regardless of contents. The user might have entered one character, or one hundred, and you're sending all 1000 bytes of whatever is at that address.

Your echo method is a bit much; I'd recommend splitting the writing off into its own routine; Advanced Programming in the Unix Environment, 2nd edition (a superb book, well worth finding a copy if you expect to program Unix or Unix-like systems) has a small sweet routine I really enjoy:

ssize_t             /* Write "n" bytes to a descriptor  */
writen(int fd, const void *ptr, size_t n)
    size_t      nleft;
    ssize_t     nwritten;

    nleft = n;
    while (nleft > 0) {
        if ((nwritten = write(fd, ptr, nleft)) < 0) {
            if (nleft == n)
                return(-1); /* error, return -1 */
                break;      /* error, return amount written so far */
        } else if (nwritten == 0) {
        nleft -= nwritten;
        ptr   += nwritten;
    return(n - nleft);      /* return >= 0 */

(See lib/writen.c in the source from the book's website.)

I haven't actually tried your code, so it's very possible none of these are the problem actually preventing your software from working :) if that's the case, leave a comment, and I'll take a closer look.

I'm reading APUE these days, and just found an issue.

One common way to use pthread_attr_setstack is to

  1. malloc a chuck of memory
  2. set the addr and length by the method

Then, here comes my question, what if I want to use the guard-size to protect my data? Say I need A bytes of memory, and B bytes for guard-size.

Shall I malloc A+B bytes, or malloc A bytes?

Actually pthread library itself provides an API to set the guard size

int pthread_attr_setguardsize(pthread_attr_t *attr, size_t guardsize);

But note that if you are setting the stack location or size, either using functions like pthread_attr_setstack(3) or pthread_attr_setstackaddr(3), which you are, then the guard size attribute is ignored (i.e., no guard area is created by the system): it is the your responsibility to handle stack overflow (perhaps by using mprotect(2) to manually define a guard area at the end of the stack that you have allocated.

So, coming to your specific question, yes, you need to malloc A+B bytes to include the guard area yourself if you wish to use pthread_attr_setstack. Else, if you are OK with the default stack size, then you use the pthread_attr_setstack function.

Read about pthreads here. Also read specifically about thread stack management here under the section Stack Management

When I open a terminal running bash and type the following:

cat\nfoo\n^Decho bar\n

(where \n is enter and ^D is control-d) I get:


Ie, ^D causes cat to exit, but I can still type more.
How would I send the same input (specifically the end-of-file) through a (unix) pipe in C?

xterm does not perform its input with the shell and utilities via pipe(7)s. Instead, it uses the Unix PTY framework (see pty(7), openpty(3), forkpty(3), posix_openpt(3), pts(4) manpages for some information). The PTY framework allows any process to serve as a terminal "master" (e.g., telnetd(8), sshd(8), xterm(1), etc.) and any process can connect to the terminal slave to provide an interactive environment just like sitting at the console.

The Advanced Programming in the Unix Environment, 2nd edition book by Stevens and Rago has an excellent chapter on using pseudo-terminal devices to control slave programs -- including an excellent little pty program that allows driving "interactive" programs in a manner similar to expect(1), but in C rather than tcl.

I am making a server and I use fork() for creating child processes but I have doubts about that. Here are some:

  1. Why do you need to close main socket in a child process and new connection accepted socket in the parent process? (after accepting a new connection) I thought that sockets are only integers with some id which are used to access opened sockets in some system-wide object which is only acceptable through system functions calls. In that case fork would only copy the integer but would have no affect on socket opened.
  2. I have checked and found out that if I fork a process inside a class method, all members are copied. Well, I have found out it is copy-on-edit so does it mean that my server class will be copied in each child that uses a non-constant function? How do I make some memory shared between all such processes (like a list of tasks and each child puts something into it while the parent is reading stuff from it)? I guess fork is not the right function for that. What is the best way?

P.S. I am pretty sure I know the answer to the second question, which is clone(), but just wanted to make sure that's the right function.

I think you might want to look through this book as a reference on fork().

  1. Yes you do need to close the socket bound to listen in the child and accepted socket in the parent. The integers aka file handles point to real structures see this so unless you want the kernel to dump a new connection on a child or parent being able to send the data to the connected client you might want to prevent this outright.
  2. To share data for between the processes the best way is shared memory. The book I referred you to will have quite a bit of information regarding that too. In general if you need to share memory without shared memory then you might want to look at threads.

P.S. I'm not sure which clone() method you are referring to. Object copying is done via copy constructors.

I wrote this program in C++ and on Linux platform.

I wrote a client and server socket program.

In that client program, I wrote socket function and immediately after that I am doing some other actions not at all depending on socket (I wrote 2 for loops for some other logic).

After that I prepared the structures required for the socket and I wrote the connect that I am getting error as unable to connect because connect is returning -1..

But for the same program, if I write that for loop's logic above the socket function and immediately after that structures, and connect function, then it is working fine..

What might be the reason I am not able to get? Please help me in this aspect. Here is my code

here index1 and index 2 are simple integer variables..The configstring is a char array contains and port number)...address and port are char array variables to store address and port number..

struct sockaddr_in s1;

    address[index1] = configstring[index1];

    port[index2] = configstring[index1];

int port_num = changeto_int(port);

if((sock_fd = socket(AF_INET,SOCK_STREAM,0)) == -1)
    printf("unable to create a socket\n");
    return 0;

memset(s1.sin_zero, '\0', sizeof s1.sin_zero);
int errno;

if(connect(sock_fd,(struct sockaddr *)&s1,sizeof(s1)) == -1)
    printf("error:unable to connect\n");
    printf("Error in connect(): %s\n", strerror( errno));
    return -1;

Once in a while you run into a book that stands head, shoulders and torso above all of the other books out there.

If you REALLY want to understand networking at the OS level (which is where networking "happens"), get a copy of Advanced Programming in the UNIX(R) Environment. It gives you the "guts", "history", and "howto" in regards to programming on UNIX, with much content dedicated to networking and IPC.