Operating Systems

Andrew S. Tanenbaum, Albert S. Woodhull

Mentioned 15

Featuring an introduction to operating systems, this work reflects advances in OS design and implementation. Using MINIX, this book introduces various concepts needed to construct a working OS, such as system calls, processes, IPC, scheduling, I/O, deadlocks, memory management, threads, file systems, security, and more.

More on Amazon.com

Mentioned in questions and answers.

This might be in vain, as I know writing an operating system is unbearably complicated (especially by oneself).

  • I don't expect to build the next linux, or windows.

  • I know it will be horrible, and buggy, and won't work, but that's fine.

I want to write everything myself, in Assembly, C, and (some) C++.

This is a future project, as I'm busy with some other things at the moment and don't have the time immediately, but I figured I would ask it now, so maybe I could get lots of answers to this, and it could build up and be a useful resource for this kind of approach (everything else I have seen involved building off of minix, using an existing bootloader, building it in a virtual booting program thing, etc).

I want to set up one of my older desktops with a monitor, keyboard and mouse, and start working on a blank hard drive.

I want to learn how to write my own bootloader (I've found lots of resources about this, but for completeness, please still add some good ones), my own USB driver (if that's necessary), a CD driver (if that's necessary), etc. Everything, from the ground up.

  • How do I put the code onto the computer? Is it best to do it with a floppy disk? Can most computers do it from a USB stick?

  • What drivers do I need, and can you suggest any references to building those?

  • After the booting sequence--then what? How do I get into protected mode etc.

  • How do I manage memory without the help of an operating system? Do I just use whatever addresses I want? No initialization necessary?

  • What will I undoubtedly run into that will confuse me?

  • How can I make it either a command line O/S, and a graphical one?

  • What is a graphical O/S built on? Like, how would I do something like, a command line, with a font, and a picture at the top?

  • Where can I read about setting up a multitasking environment? (ie., having two graphical-like command lines running side-by-side).

  • How would I set up a sort of windowing system? How do I display graphics on the screen once simple multitasking is set up?

Believe me, I understand that this is a very complicated project, and I probably will never get around to completing it or writing anything on it of any use.

There are lots of other pieces to this I haven't mentioned, if you think of any, feel free to add those too.

Please put one "topic" per answer--for example, USB drivers, and then maybe a list of resources, things to look out for, etc.

Also, please don't suggest building off of another O/S or pre-existing code. I know I will read a lot of pre-existing code (such as the linux kernel, or example resources, existing drivers, etc) but ultimately I want to do all the writing myself. I know I should build off of something else, and there are lots of other questions on SO about that that I can read if I change my mind and go that route. But this one is all about doing the whole thing from scratch.

update

I've got lots of great answers to this, mostly about the booting process, file systems and various existing projects to read for reference.

Any suggestions on how to get it graphical? Different video modes and how to work with them, etc?

Take a look at Minix. Study the source code along with "Operating Systems Design and Implementation". Consider making contributions to the project. I think Minix is a really good and promising OS in the making. It is also well funded project. That means, you might even get paid for your contributions!

First things first. Read, read, read, read, read. You need to have a firm understanding of how the OS works before you can hope to implement your own.

Grab one of Andrew Tanenbaum's books on operating systems. This is the one we used in my OS class in college:

Modern Operating Systems

Modern Operating Systems on Amazon

Despite the ridiculous cover, it's a fantastic read, especially for a textbook. Tanenbaum is really an expert in this area and his explanations of how the OS works underneath the hood are clear and easy to understand. This book is mostly theory, but I believe he also has a book that discusses more of the implementation. I've never read it, though, so I can't comment on it.

That should help you bone up on process management, memory management, filesystems, and everything else your OS kernel needs to do to get it up to a bootable state. From that point on it's basically a matter of writing device drivers for the hardware you need to support, and offering implementations of the C library functions to make kernel calls for things like opening files and devices, reading and writing, passing messages between processes, etc.

Read up on x86 assembly (assuming you are designing this for an x86 machine). That should answer a lot of your questions with regards to moving between processor operating modes.

If you've got any electronics knowledge, it may be easier to start with writing an operating system for an embedded device that has ample documentation, because it will generally be simpler than an x86 PC. I've always wanted to write my own OS as well, and I'm starting with writing a microkernel embedded OS for this development board from Digilent. It can run the soft-core MicroBlaze processor from Xilinx, which has very thorough documentation. It's also got some RAM, flash data storage, LEDs, switches, buttons, VGA output, etc. Plenty of stuff to play around with writing simple drivers for.

One of the benefits of an embedded device is also that you may be able to avoid writing a VGA driver for a long time. In my case, the Digilent development board has an onboard UART, so I can effectively use the serial output as my console to get the whole thing up and booting to a command line with minimal fuss.

Just make sure that whatever you choose to target has a readily available and well-tested compiler for it. You do not want to be writing an OS and a compiler at the same time.

I'm planning to write an operating system and I don't know very much about operating systems. Are there any good resources or books to read in order for me to learn? What are your recommendations?

Operating System Concepts is the book we used at University. It's quite ugly BUT the information inside are well explain (from basic memory management, to how to OS decide what to execute or how to avoid deadlock). Pretty wide.

alt text

While old, these books are very good:

Operating System Design with Xinu

alt text

Operating System Design-Internetworking With XINU, Vol. II

alt text

3: http://Operating System Design-Internetworking With XINU, Vol. II

Operating Systems Implementation Prentice Software

alt text

This book is written by Tanenbaum, the main guy behind Minix, which is what Linux was based on. It provides good overviews for basic OS concepts like memory management, file systems, processes, etc. The concepts in this book book are intimately tied to examples of the Minix OS, which is a good thing.

I think you should start by something like that.

We used Andrew Tannenbaum's Modern Operating Systems at the university I attended. I highly recommend it for it's clear explanations of the tradeoffs inherent in many of the design decisions that you'll run up against. This book is a little bit more "fair and balanced" than the Minix book.

alt text

I also recommend this book because, despite his net-famous flame war with Linus Torvalds, few of his biases come through in the book. Also, he's a pretty decent writer, and the book is actually entertaining.

Creating an OS seems like a massive project. How would anyone even get started?

For example, when I pop Ubuntu into my drive, how can my computer just run it? (This, I guess, is what I'd really like to know.)

Or, looking at it from another angle, what is the least amount of bytes that could be on a disk and still be "run" as an OS?

(I'm sorry if this is vague. I just have no idea about this subject, so I can't be very specific. I pretend to know a fair amount about how computers work, but I'm utterly clueless about this subject.)

I can't believe this hasn't been mentioned... but a classic book for an overview of operating system design is Operating Systems - Design and Implementation written by Andrew S Tanenbaum, the creator of MINIX. A lot of the examples in the book are geared directly towards MINIX as well.

If you would like to learn a bit more, OS Dev is a great place to start. Especially the wiki. This site is full of information as well as developers who write personal operating systems for a small project/hobby. It's a great learning resource too, as there are many people in the same boat as you on OSDev who want to learn what goes into an OS. You might end up trying it yourself eventually, too!

I love to code, but I am currently only doing web development. I'd like to do something that will be unique and fun and very different from web programming.

Yeah, this might be a dumb question, but I think it would be really cool to build a really simple Operating System. So please do not say anything rude. I just want to know the following things:

*Where to start? *Resources *What language would I use?

I was thinking something simple like a cmd based

Start by reading some operating system books - like Tanenbaums' Modern operating systems.

This should give you an understanding of what problems you need to solve in order to write an operating system.

The absolute "bible" on operating system design is and was Andrew Tanenbaum's Operating Systems Design and Implementation - the "Dragon" book :-)

alt text

There are plenty of other references, too, e.g. Developing Your Own 32-Bit Operating System.

Microsoft Research also has/had a project on creating an operating system in managed code (C#) called Singularity - that might provide some insights/papers etc.

Writing a complete OS is neither a trivial nor a quick task, though.....

I'd like to gain better knowledge of operating system internals. Process management, memory management, and stuff like that.
I was thinking of learning by getting to know either linux or BSD kernel.
Which one kernel is better for learning purposes?
What's the best place to start?
Can you recommend any good books?

There's no substitute for diving into the code. Try to find a driver or subsystem that you're interested in and poke around with it. With tools like VMware Workstation it's super easy to make whatever changes you want, snapshot the VM, and run your modified kernel. If the kernel panics on boot, who cares? Just jump back to the snapshot and fix the problem.

For books, I strongly recommend Linux Kernel Development by Robert Love. It's a wonderfully written book -- lots of information, organized sanely, and humorous... not dry reading at all.

Noting the lack of BSDs here, I figured I'd chip in:

I haven't taken any of the courses myself, but I've heard Marshall Kirk McKusick speak on other occasions, and he is really good at what he does.

And of course the BSD man pages, which are an excellent resource as they are maintained to a far greater extent than your average Linux man-page. Take for instance the uvm(9) man-page, describing the virtual memory interface in OpenBSD.

Not quite related, but I'll also recommend the video History of the Berkeley Software Distributions as it gives a nice introduction to the BSD parts of the UNIX history and culture as well as plenty of hilarious anectodes from back when.

I had previously bought these books on recommendation for the same purpose but I never got to studying them myself so only take them as second-hand advice.

In college, I had an operating systems class where we used a book by Tanenbaum. In the class, we implemented a device driver in the Minix operating system. It was a lot of fun, and we learned a lot.

One thing to note though, if you pick Minix, it is designed for learning. It is a microkernel, while Linux and BSD are a monolithic kernel, so what you learn may not be 100% translatable to be able to work with Linux or BSD, but you can still gain a lot out of it, without having to process quite as much information.

As a side note, if you've read Just for Fun, Linus actually was playing with Minix before he wrote Linux, but it just wasn't enough for his purposes.

What are some tiny open source operating systems? I'm looking for something several orders of magnitude smaller than Puppy Linux, Feather Linux, DSL, etc. I want to run a command-line text editor and compiler; anything else is extraneous. I'm looking for a system I can take apart and acquire a fairly good understanding of the whole thing in a reasonable period of time. Bonus points for something that is portable and well-documented.

I'll agree on Minix. Also the book:

http://www.amazon.com/Operating-Systems-Implementation-Prentice-Software/dp/0131429388

is actually fairly comprehensible. (Unlike some books on similar topics, thinking of "Understanding the Linux kernel" here.) It's quite expensive, but you can probably pick up an earlier edition second-hand.

(NB. I'm not in any way affiliated with the book, it's just good :) )

OpenBSD could be a good choice, I think the default install (no X11) has something like 15 processes. FreeBSD is a close second. Either way you'll want to read "The Design and Implementation of the FreeBSD Operating System"

BTW, I've spent a considerable amount of time on AIX, Solaris, FreeBSD, and Linux... and I think the OpenBSD man pages are the best around.

What kind of C++(Restate to programming) would I have to learn to make my own OS kernel? (I know I would also have to learn assembly.) EDIT***Like interrupts , keyboard driver, getting input.***

Ok everyone I made a really * 3 basic OS and would like to share it.

Here you go. http://bcsy.hostzi.com/BytOS.zip

Compile on linux

check out nanoos, its an OS project in C++!

In answer to your question:

An operating system (commonly abbreviated to either OS or O/S) is an interface between hardware and applications; it is responsible for the management and coordination of activities and the sharing of the limited resources of the computer. The operating system acts as a host for applications that are run on the machine. So you need to learn the bits in C++ that enable your program (OS) to interact with the hardware it will be running on.

Then it needs to go beyond interacting with the hardware and providing use to the user.

For that, go back once again and see what is being attempted in the nanoos as a starting point (for C++ learning needs of what building an OS in C++ would entail)

Features found in nanoos:

  • C++ run time

  • 32-bit protected mode Memory manager

  • CPU detection, Memory detection and IDE hard disk detection

  • IDE hard disk read

Basically, to write an Operating System, just check out what an OS needs to do (or what it is)

If I was to write an OS, before even considering the language, I would try to understand what an OS is, this is a great book: Operating Systems Design and Implementation.

alt text

Have a read. Good luck

I have to choose a thesis topic soon and I was considering implementing an operating system for an architecture that is not x86 (I'm leaning towards ARM or AVR). The reason I am avoiding x86 is because I would like to gain some experience with embedded platforms and I (possibly incorrectly) believe that the task may be easier when carried out on a smaller scale. Does anyone have any pointers to websites or resources where there are some examples of this. I have read through most if not all of the OSDev questions on stack overflow, and I am also aware of AvrFreaks and OSDev. Additionally if anyone has had experience in this area and wanted to offer some advice in regards to approach or platform it would be much appreciated.

Thanks

Seems like you should get a copy of Jean Labrosse's book MicroC/OS.

It looks like he may have just updated it too.

http://micrium.com/page/press_room/news/id:40

http://micrium.com/page/home

This is a well documented book describing the inner workings of an RTOS written in C and ported to many embedded processors. You could also run it on a x86, and then cross compile to another processor.

If you choose ARM, pick up a copy of the ARM System Developer's Guide (Sloss, Symes, Wright). Link to Amazon

Chapter 11 discusses the implementation of a simple embedded operating system, with great explanations and sample code.

I looked at some other questions on SO and its not clear if c is built on top of, under, or alongside the WINAPI. Like for example could someone write something in pure c that was able to open a window, or would they need to use the windows api?

I noticed similarities between the c (library?) version of opening a file (fopen) vs the windows API version (CreateFile) which makes me wonder if one is just a wrapper for the other. Does anyone know?

If windows is running; is a programmer forced to program using the windows api to get something running on it or can the programmer not use the windows api at all and directly access the hardware (ie. does the windows operating system protect access to the hardware)?

Which is more portable between different versions of windows of windows ce. The documentation I found (which has now changed) used to say that CreateFile only goes back to version 2.0 of windows ce (here: http://msdn.microsoft.com/en-us/library/ms959950.aspx - Notice the note on the link at the very bottom that shows the supported version information has been changed). So what is one supposed to use for windows ce version 1? In other words is programming using c functions or the functions labeled WINAPI more likely to work on all versions of windows CE?

I read the following in a book about programming windows ce and it confused me, so all of the above questions can be understood better in context of making sense of the following:

Windows CE supports the most of the same file I/O functions found on Windows NT and Windows 98. The same Win32 API calls, such as CreateFile, ReadFile, WriteFile and CloseFile, are all supported. A Windows CE programmer must be aware of a few differences, however. First of all, the standard C file I/O functions, such as fopen, fread, and fprintf, aren't supported under Windows CE. Likewise, the old Win16 standards, _lread, _lwrite, and _llseek, aren't supported. This isn't really a huge problem because all of these functions can easily be implemented by wrapping the Windows CE file functions with a small amount of code.

My understanding of wrapping is that you have to have something to wrap, given how it appears that win16 and c library are not available is he stating to wrap the CreateFile function to make your own c-like version of fopen? (The only other thing I am aware of is assembly, and if that is what he was suggesting to wrap it wouldn't be written in such a casual manner.)

Given the above, what is the dependency relationship between c language (syntax, data structures, flow control), the c function library (ex. fopen), and the windows api (ex. CreateFile)?

Most operating systems, including Windows are written in C (and or assembler). The Library is then modified for each operating system to do the basic stuff. (Sockets, Files, Memory, etc ...).

The WINAPI is just a bunch of libraries (written in C and/or Assembler) that allow access to functionality within the OS.

It is not Windows related, after you changed your question, I think what you are trying to understand is the bootstrapping of an OS (Windows or other).
The book Operating Systems Design and implementation discusses the implementation of Minix (Which Linux is based on).

Hi all I'm looking for a way to create a very small subset of an Operating System in Microsoft Macro Assembler(MASM). Preferably a system that can at basic boot a standard pc and give a directory listing; any resource or advice that can point me in the right direction will be welcomed. Thanks in advance.

I think Operating Systems Design and Implementation could satisfy most of your questions, what I don't really know if MASM is the compiler chosen for low level parts. Most of the code however is C based with an small percent of asm. However, directory listing depends on the filesystem you want to read from so, that's something else, but it's covered there too.

I am making an OS from scratch. I have made a hello world bootloader using the tutorials from the internet.

I am stuck at the next step though.

I need to call an executable ( basically my shell code from the ASM bootloader code ). I googled around and couldn't find anything substantial that was explained in a good way.

So, can someone help me connect my bootloader to the shell? A small snippet of code as an example would be great ( I understand better by code ). I promise, I would use that only to understand and not copy anything.

Thanks!

The part which goes between the bootloader and an application (such as a shell) is kind of... substantial. One could say that "making an OS from scratch" actually means writing that part, often called the kernel; compared to that, the bootloader is very small and simple.

Let's take Linux as example (on 32-bit x86). A Linux shell is a collection of x86 opcodes which expect to be loaded at a fixed address in RAM (that was chosen when the shell was compiled, or, more accurately, linked). The first job of the kernel is to setup the MMU so that the shell will have that view of the memory. The shell will communicate with the outside world by invoking the kernel, and it will do so through system calls. In Linux-x86, system calls use the int opcode (this triggers a software interrupt, and the system call arguments are conventionally passed in some of the registers). Some of the important system calls include the read and write calls: from the point of view of the shell, what you type on the keyboard can be read from a virtual file, indexed by a descriptor (an integer, with value 0 for "standard input"). The job of the kernel here is to accumulate key strokes (each will trigger a hardware interrupt, that the kernel receives -- the kernel is supposed to respond to hardware interrupts -- and translates into characters) and return them to the application when the application asks for them. Similarly, data written out by the shell on what the shell thinks of as "standard output" (descriptor 1) must be translated by the kernel into characters to be displayed, which implies talking to the video hardware.

A great resource of learning out to build an operating system is Minix. Minix was initially meant as a learning tool, described at length in Tannenbaum's book "Operating Systems Design and Implementation". It is opensource and free. Do yourself a favor, download Minix source code and buy the book (or borrow it from a library somewhere).

I have good knowledge of C++ and after reading The Elements of Computing Systems I have a basic knowledge of how computers work. I made a list of topics I want to learn next and books I want to buy on those topics. One of them is operating systems, but the one on the top of the list is Game development.

I am still a noob on those topics, so I wonder if I should know how an operating system (unix specifically) works before trying to learn game programming (Opengl, etc). On operating systems I have the book Operating Systems by Tanenbaum, and I want to buy The Linux Programming Interface by Michael Kerish.

On game development I was planning on buying Game Engine Architecture and Game Coding Complete to acquire a general concept of game programming and how engines work and then learn Opengl.

I am really lost on what to do first and I hope this is an appropriate question. What should I learn first, what books should I read and in what order. Should I learn how a VGA works before trying Opengl? Are there any other topics I should know before delving into games programming. I am asking this because I like to know what I am coding, what the functions I am calling do under the hood, I don't like holes in my knowledge.

Thanks.

Fluffy opinion answer incoming. Take with grain of salt.

The nice thing about programming is that that you don't need to learn everything about everything to do one thing effectively. Knowing exactly how to implement a video driver isn't required for using OpenGL effectively. The point of OpenGL is to abstract that out so you don't have to worry.

Since you want to do game development, make a project. Like recreating Asteroids using OpenGL for graphics and writing all the game logic yourself. And set about completing it. In the process you'll learn much more than simply reading. Use books as reference. At least thats what I've found works for me.

The Operating Systems book is pretty good. Its the one I read in college. But those concepts presented in it, though interesting, are not something you'll have trouble learning simultaneously with game development or anything else.

Also you should read this: http://www.linuxforu.com/tag/linux-device-drivers-series/. It's a great article series that teaches linux driver development and operating systems concepts in the process.

I wish to make a custom kernel, that loads and executes code (my code is similar to lua) so that I can use it to make an OS, I have already made a basic kernel in java, but I really need to know how to make it in Assembly.

Can anyone give me the steps to making an efficient kernel in assembly?

If you just want to fiddle around a bit, look at this guide.

You could also look through the code of this project.

A popular book on this matter is this one.

I found one here http://mjc88.0catch.com/ but then some files and description is missing. Please direct me to one, complete open source OS, which is realistic to be build by a single person and i can extend it with my ideas.

One popular choice for this sort of thing is uC/OS. If you buy the book you get a CD with all the source code etc.

You can have a look at that book: Tanenbaum: Operating Systems Design And Implementation