Windows Via C/C++

Jeffrey Richter, Christophe Nasarre

Mentioned 21

"Get the classic book for programming Windows in Microsoft Visual C++--now in its fifth edition and fully revised for Windows Vista. This must-have book covers programming at the API level with code samples in Visual C++"--Resource description p.

More on Amazon.com

Mentioned in questions and answers.

Rebasing a DLL means to fix up the DLL such, that it's preferred load adress is the load address that the Loader is actually able to load the DLL at.

This can either be achieved by a tool such as Rebase.exe or by specifying default load addresses for all your (own) dlls so that they "fit" in your executable process.

The whole point of managing the DLL base addresses this way is to speed up application loads. (Or so I understand.)

The question is now: Is it worth the trouble?

I have the book Windows via C/C++ by Richter/Nazarre and they strongly recommend[a] making sure that the load addresses all match up so that the Loader doesn't have to rebase the loaded DLLs.

They fail to argue however, if this speeds up application load times to any significant amount.

Also, with ASLR it seems dubious that this has any value at all, since the load addresses will be randomized anyway.

Are there any hard facts on the pro/cons of this?

[a]: In my WvC++/5th ed it is in the sections titled Rebasing Modules and Binding Modules on pages 568ff. in Chapter 20, DLL Advanced Techniques.

I'd like to provide one answer myself, although the answers of Hans Passant and others are describing the tradeoffs already pretty well.

After recently fiddling with DLL base adresses in our application, I will here give my conclusion:

I think that, unless you can prove otherwise, providing DLLs with a non-default Base Address is an exercise in futility. This includes rebasing my DLLs.

  • For the DLLs I control, given the average application, each DLL will be loaded into memory only once anyway, so the load on the paging file should be minimal. (But see the comment of Michal Burr in another answer about Terminal Server environment.)

  • If DLLs are provided with a fixed base address (without rebasing) it will actually increase address space fragmentation, as sooner or later these addresses won't match anymore. In our app we had given all DLLs a fixed base address (for other legacy reasons, and not because of address space fragmentation) without using rebase.exe and this significantly increased address space fragmentation for us because you really can't get this right manually.

  • Rebasing (via rebase.exe) is not cheap. It is another step in the build process that has to be maintained and checked, so it has to have some benefit.

  • A large application will always have some DLLs loaded where the base address does not match, because of some hook DLLs (AV) and because you don't rebase 3rd party DLLs (or at least I wouldn't).

  • If you're using a RAM disk for the paging file, you might actually be better of if loaded DLLs get paged out :-)

So to sum up, I think that rebasing isn't worth the trouble except for special cases like the system DLLs.


I'd like to add a historical piece that I found on Old New Thing: How did Windows 95 rebase DLLs? --

When a DLL needed to be rebased, Windows 95 would merely make a note of the DLL's new base address, but wouldn't do much else. The real work happened when the pages of the DLL ultimately got swapped in. The raw page was swapped off the disk, then the fix-ups were applied on the fly to the raw page, thereby relocating it. The fixed-up page was then mapped into the process's address space and the program was allowed to continue.

Looking at how this process is done (read the whole thing), I personally suspect that part of the "rebasing is evil" stance dates back to the olden days of Win9x and low memory conditions.


I have experience writing console and network client/server applications in C and C++, but I know next to nothing about using the win32 visual API, MFC, Qt, wxWidgets, etc. Where is a good place to start, and what method should I specialize in, so as to be future ready and robust?

This is a rather broad question, as programming GUI applications in Windows can be done in so many ways.

There are two main parts to developing any GUI app: the language and the API/framework. Considering you're interested in learning to build Windows GUI apps, the language isn't really a point of focus for you. Hence, you should pick a language you already know and work with a framework or API that can be harnessed by your chosen language.

If you want to use C you're pretty much restricted to dealing with the Win32 API yourself, in which case reading Petzold or Richter would be great places to start. The Win32 API can be quite daunting, but it's well worth the effort to learn (imho). There are plenty of tutorials on Win32 on the web, and there's always MSDN, with a complete reference/guide to the Win32 API. Make sure you cover not just the API, but other areas such as resources/dialogs as they are building blocks for your Win32 application.

If you want to use C++ you have all of the options that you have when using C plus a few others. I'd recommend going with the Win32 API directly, and then moving on to a known framework such as MFC, Qt, wxWindows or GTK so that you can spend less time working with boilerplate code and instead focus on writing your application logic. The last 3 options I just listed have the added benefit of being cross-platform, so you don't have to worry too much about platform-specific issues. Given that you said you want to work with Windows, I'll assume you're keen to focus on that rather than cross-platform -- so go with MFC, but spend some time with the Win32 API first to get familiar with some of the concepts.

When dealing with MFC and the Win32 API, it's a good idea to try and get a solid understanding of the terminology prior to writing code. For example, you need to understand what the message pump is, and how it works. You need to know about concepts such as "owner-drawn controls", and subclassing. When you understand these things (and more), you'll find it easier to work with MFC because it uses similar terminology in its class interfaces (eg. you need to know what "translate messages" means before you can understand how and when to use PreTranslateMessage).

You could also use Managed C++ to write .NET GUI applications, but I've read in a few places that Managed C++ wasn't really intended to be used in this manner. Instead it should be used as a gateway between native/unmanaged code and managed code. If you're using .NET it's best to use a .NET language such as VB.NET or C# to build your GUIs.

So if you are going to use .NET, you currently have the choice of the WinForms library, or WPF. I personally feel that you'd be wasting time learning to build WinForms applications given that WPF is designed to replace it. Over time WPF will become more prevelant and Winforms will most likely die off. WPF has a much richer API set, and doesn't suffer from many of the limitations that Winforms does. If you do choose this route, however, you'll no doubt have to learn XAML, which is a markup language that drives WPF applications. This technology is coming of age, and there are many great places to learn about it. First, there are sites such as LearnWPF, and DrWPF which have some really great articles. Secondly, there are plenty of quality books on the topic.

So, to sum up, once you've picked your language and tech, the path is actually quite easy. Just pick up a book or two, read some blogs, get into some code samples.. and most importantly ... write code. Keep writing, keep making mistakes, and keep learning from them.

As a final note...

In other words, Silverlight. If you don't want to go the MS route you might give Adobe's Flash/Flex a look see. Both Silverlight and Flash/Flex build RIA's. Which I think is where we are headed. They days of Office like apps are numbered

I don't agree at all. Silverlight is not the same as WPF. Silverlight is web-specific, and only has a subset of WPF's features. Given that the question asks for Windows GUI apps, Flash/Flex Rich Internet Apps are not really a fitting suggestion. I also don't agree that the days of Rich Client Applications (such as office) are numbered at all.

I hope that helps. Good luck :)

I have very little experience building software for Windows, and zero experience using the Windows API, but I'm reasonably familiar with Python. So, how should I go about learning to use the Windows API with Python?

You have to start with these two books.

Petzold book: Great for learning messages and message pumps, GDI and User32 stuff.

Richter book: The windows base services, viz. Processes, memory, threads and dlls

I was going through the Hilo tutorial series Developing C++ Applications for Windows 7; seemed pretty interesting.

What modern books that go into details of developing C++ based applications for Windows 7? It should show how to take advantage of Windows 7 features and based on "modern" C++ (templates, Unicode, etc.). Not looking for old school Petzold or MFC type books (sorry). Also should feature native code development (i.e. no Qt-/wx-type 3rd party libraries). The 3rd party libraries seems to be at least a generation behind and don't seem to leverage the latests features (ex. Ribbon, Animation etc.).

I don't know if it's still 'modern', but Modern C++ Design is a great resource on templates and generic algorithms.

I'd recommend that you ensure you're familiar with the theory in winprog.org/tutorial, then Ivor Horton's book (below), and then finally check out the WTL library.

The examples on the ribbon code in the Hilo articles use the windows API, which relies on COM objects to create the ribbon. ATL constructs like the CComPtr manage the lifetime of the ribbon objects and interfaces.

Because of this, I'd recommend that you ensure that you understand the basics of writing COM client code - e.g. You should understand CoCreateInstance, QueryInterface and smart pointers. ATL Internals (search on Amazon) is the essential reference on COM, but it is heavy going.

WTL wraps the detail of creating the ribbon and accessing it, but WTL was made for COM programmers looking for a windowing API, so it's not an easy API to begin with if you don't have experience of COM programming with ATL.

WTL supports the latest windows features, like ribbons etc. Don't expect any books on the subject though, there aren't any. M$ open sourced WTL some years ago, so the code is available to read, so you can figure out how everything works by reading the code. It's quite advanced though, you may want to look at it after you've had a look at the alternatives first.

Google uses WTL for the UI for Chrome.

I suspect you should Start with Ivor Horton's beginning Visual C++ 2010, it'll be a good all round introduction.

If you're looking for a great programming book for windows though, you can't do better than windows via c/c++ by Jeff richter - by far the best windows coding book I've ever bought.

If you are interested in WTL, then look at Michael dunn's series wtl for mfc programmers on code project.

Finally, there is no one book that hand hold the would be c++ GUI developer through all this, and that's a big gap in the market. I've often wondered why no one has picked up on this, given the popularity of c++

I'd like to learn programming with windows libraries. Could You recommend me some technical books / tutorials. I'm not looking for a book for beginers, lets asume that the reader knows C++/OOP/STDLIB/STL.

Topics I need to be covered are ~ WinAPI (Base/Advanced Services), MFC, COM, ATL and possibly WTL if it makes sense. I'm not trying to learn .NET,C# and so on.

Thanks for help.

For Win32 nothing beats Programming Windows by Petzold and Windows via C/C++ by Ricther for COM, certainly Essential COM by Don Box and there is even a new edition of ATL Internals. Once you know the Win32 API the MFC layer on top of that is mostly learnable via MSDN, so I don't have a good recommendation there.

The topics you are interested in are generally too broad to be covered in depth by a single book. Having said that, you can check this one: http://www.amazon.com/Visual-NET-Bible-Tom-Archer/dp/0764548379 - it covers most of the topics you are interested in, even if it is a bit outdated.

In a Win32 C++ application, we start a message loop that fetches messages from a queue, translates them and then dispatches them. Eventually, each message reaches our WndProc where the associated event can be handled.

I understand that part. What I don't understand is the in between goings-on. Specifically:

  1. Different kinds of OS interrupt handlers must be placing messages in the said 'message queue', but where within the process address space does this queue reside? How is it exposed to the interrupt handler code?
  2. What does it mean to 'translate' the message? What does the call to TranslateMessage() really do?
  3. Once dispatched by DispatchMessage(), what all places does the message swing by before reaching my WndProc (i.e. what does the OS do with it)?

If anyone knows the answers to the above, kindly satisfy my curiosity. Thanks.

Different kinds of OS interrupt handlers must be placing messages in the said 'message queue', but where within the process address space does this queue reside? How is it exposed to the interrupt handler code?

Windows are associated with threads. Each thread with a window has a thread queue in the process's address space. The OS has an internal queue in its own address space for the hardware-generated events. Using details of the event and other state information (e.g., which window has the focus), the OS translates the hardware events into messages that are then placed in the appropriate thread queue.

Messages that are posted are placed directly in the thread queue for the target window.

Messages that are sent are usually processed directly (bypassing the queue).

The details get hairy. For example, the thread queues are more than lists of messages--they also maintain some state information. Some messages (like WM_PAINT) aren't really queued, but synthesized from the additional state information when you query the queue and it's empty. Messages sent to windows owned by other threads are actually posted to the receiver's queue rather than being processed directly, but the system makes it appear like a regular blocking send from the caller's point of view. Hilarity ensues if this can cause deadlock (because of circular sends back to the original thread).

The Jeffrey Richter books have a lot (all?) of the gory details. My edition is old (Advanced Windows). The current edition seems to be called Windows via C/C++.

The OS does a LOT of work to make the message stream appear rational (and relatively simple) to the caller.

What does it mean to 'translate' the message? What does the call to TranslateMessage() really do?

It watches for virtual key messages and, when it recognizes a key-down/key-up combination, it adds character messages. If you don't call TranslateMessage, you won't receive character messages like WM_CHAR.

I suspect it sends the character message directly before returning (as opposed to posting them). I've never checked, but I seem to recall that the WM_CHAR messages arrive just before the WM_KEYUP.

Once dispatched by DispatchMessage(), what all places does the message swing by before reaching my WndProc (i.e. what does the OS do with it)?

DispatchMessage passes the message to the WndProc for the target window. Along the way, it some hooks may get a chance to see the message (and possibly interfere with it).

I think of myself as a pretty decent developer, however when it comes to multithreading I'm a total n00b. I mean the only multithreading I've done at work was the very basic stuff like spawning off multiple threads using the ThreadPool to do some background work. No synchronization was necessary, and there was never any need to create threads manually.

So, my question is this; I want to write some application that will need to be heavily multithreaded and that will need to do all of the advanced things like synchronization etc.. I just can't think of anything to write. I've thought of maybe trying to write my own ThreadPool, but I think I need to learn to walk before I can run. So what ideas can anyone suggest? It doesn't have to have any real world useage, it can be totally pointless and worthless, but I just want to get better. I've read tons of articles and tutorials on all the theory, but the only way to REALLY get better is by doing. So, any ideas?

I think you should draw attention for this books:

  1. Windows via C/C++ by Jeffrey Richter. This is one of the best books about multithreading
  2. Concurrent Programming on Windows by Joe Duffy. Another excelent book about multithreading

Excelent articles by Herb Sutter (Herb, we all waiting for your new book!)

Effective Concurrency series

Some blogs:

  1. Herb Sutter's blog.
  2. Parallel programming with .Net
  3. Jeffrey Richter's Blog
  4. Joe Duffy's blog

P.S. How about Power Threading as an example of multithreading (and ThreadPool implementation)?

I have been a .NET developer since I started coding. I would like to learn Win32 programming. Need advice on where to start. What are the best resources /books for learining Win32 programming. I know a bit 'college C++'.

If you are interested in UI development, the best book for direct Win32 development in C or C++ (no MFC) is Programming Windows by Charles Petzold

For other sorts of Win32 development, such as threading, memory, DLL's, etc., Windows via C/C++ by Jeffrey Richter is a great book.

For general Windows architecture, Windows Internals by David Solomon and Mark Russinovich is a great resource.

While you're at it, pick up this book:

C++ Pointers and Dynamic Memory Management

It's old (circa 1995) but it's one of the best books to demystify pointers. If you ever found yourself blindly adding *'s or &'s to get your code to compile you probably need to read this.

I would like to run arbitrary console-based sub-processes and manage them from a single master process. The console based sub-processes communicate via stdin, stdout and stderr, and if you run them in a genuine console they terminate cleanly when you press CTRL+C. Some of them may in fact be a tree of processes, such as a batch script that runs an executable which may in turn run another executable to do some work. I would like to redirect their standard I/O (for example, so that I can show their output in a GUI window) and in certain circumstances to send them a CTRL+C event so that they will give up and terminate cleanly.

The following two diagrams show first the normal structure - one master process has four worker sub-processes, and some of those workers have their own subprocesses; and then what should happen when one of the workers needs to be stopped - it and all of its children should get the CTRL+C event, but no other processes should receive the CTRL+C event.

Process diagram

Additionally, I would much prefer that there are no extra windows visible to the user.

Here's what I've tried (note that I'm working in Python, but solutions for C would still be helpful):

  • Spawning an extra intermediate process with CREATE_NEW_CONSOLE, and then having it spawn the worker process. Then have it call GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) when we want to kill the worker. Unfortunately, CREATE_NEW_CONSOLE seems to prevent me from redirecting the standard I/O channels, so I'm left with no easy way to get the output back to the main program.
  • Spawning an extra intermediate process with CREATE_NEW_PROCESS_GROUP, and then having it spawn the worker process. Then have it call GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) when we want to kill the worker. Somehow, this manages to send the CTRL+C only to the master process, which is completely useless. On closer inspection, GenerateConsoleCtrlEvent says that CTRL+C cannot be sent to process groups.
  • Spawning the subprocess with CREATE_NEW_PROCESS_GROUP. Then call GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, pid) to kill the worker. This is not ideal, because CTRL+BREAK is less friendly than CTRL+C and will probably result in a messier termination. (E.g. if it's a Python process, no KeyboardInterrupt can be caught and no finally blocks run.)

Is there any good way to do what I want? I can see that I could theoretically build on the first attempt and find some other way to communicate between the processes, but I am worried it will turn out to be extremely awkward. Are there good examples of other programs that achieve the same effect? It seems so simple that it can't be all that uncommon a requirement.

I don't know about managing/redirecting stdin et. al., but for managing the subprocess tree have you considered using the Windows Job Objects api?

There are several other questions about managing process trees (How do I automatically destroy child processes in Windows? Performing equivalent of “Kill Process Tree” in c++ on windows) and it looks like the cleanest method if you can use it.

Chapter 5 of Windows Via C/C++ by Jeffery Richter has a good discussion on using CreateJobObject and the related APIs.

Most known AntiVirus software has a real-time monitoring feature, which means it can scan files before been accessed or executed. How a technique like this can be implemented? There is something in .NET called filewatcher I don't know if this is the same idea of what is used in AntiVirus.

Hooking a (Win)API functions is a general solution for such tasks but I suppose this is just a tip of the iceberg. There is little note about it in wiki (Detection subsection). Thus, you need to learn about how to hook API functions and about Windows internals in general. I would suggest Windows via C++ as a good start point for solving this global problem.

We have an Windows32 application in which one thread can stop another to inspect its state [PC, etc.], by doing SuspendThread/GetThreadContext/ResumeThread.

if (SuspendThread((HANDLE)hComputeThread[threadId])<0)  // freeze thread
   ThreadOperationFault("SuspendThread","InterruptGranule");
CONTEXT Context, *pContext;
Context.ContextFlags = (CONTEXT_INTEGER | CONTEXT_CONTROL);
if (!GetThreadContext((HANDLE)hComputeThread[threadId],&Context))
   ThreadOperationFault("GetThreadContext","InterruptGranule");

Extremely rarely, on a multicore system, GetThreadContext returns error code 5 (Windows system error code "Access Denied").

The SuspendThread documentation seems to clearly indicate that the targeted thread is suspended, if no error is returned. We are checking the return status of SuspendThread and ResumeThread; they aren't complaining, ever.

How can it be the case that I can suspend a thread, but can't access its context?

This blog http://www.dcl.hpi.uni-potsdam.de/research/WRK/2009/01/what-does-suspendthread-really-do/

suggests that SuspendThread, when it returns, may have started the suspension of the other thread, but that thread hasn't yet suspended. In this case, I can kind of see how GetThreadContext would be problematic, but this seems like a stupid way to define SuspendThread. (How would the call of SuspendThread know when the target thread was actually suspended?)

EDIT: I lied. I said this was for Windows.

Well, the strange truth is that I don't see this behavior under Windows XP 64 (at least not in the last week and I don't really know what happened before that)... but we have been testing this Windows application under Wine on Ubuntu 10.x. The Wine source for the guts of GetThreadContext contains an Access Denied return response on line 819 when an attempt to grab the thread state fails for some reason. I'm guessing, but it appears that Wine GetThreadStatus believes that a thread just might not be accessible repeatedly. Why that would be true after a SuspendThead is beyond me, but there's the code. Thoughts?

EDIT2: I lied again. I said we only saw the behavior on Wine. Nope... we have now found a Vista Ultimate system that seems to produce the same error (again, rarely). So, it appears that Wine and Windows agree on an obscure case. It also appears that the mere enabling of the Sysinternals Process monitor program aggravates the situation and causes the problem to appear on Windows XP 64; I suspect a Heisenbug. (The Process Monitor doesn't even exist on the Wine-tasting (:-) machine or the XP 64 system I use for development).

What on earth is it?

EDIT3: Sept 15 2010. I've added careful checking to the error return status, without otherwise disturbing the code, for SuspendThread, ResumeThread, and GetContext. I haven't seen any hint of this behavior on Windows systems since I did that. Haven't gotten back to the Wine experiment.

Nov 2010: Strange. It seems that if I compile this under VisualStudio 2005, it fails on Windows Vista and 7, but not earlier OSes. If I compile under VisualStudio 2010, it doesn't fail anywhere. One might point a finger at VisualStudio2005, but I'm suspicious of a location-sensitivve problem, and different optimizers in VS 2005 and VS 2010 place the code a slightly different places.

Nov 2012: Saga continues. We see this failure on a number of XP and Windows 7 machines, at a pretty low rate (once every several thousand runs). Our Suspend activities are applied to threads that mostly execute pure computational code but that sometimes make calls into Windows. I don't recall seeing this issue when the PC of the thread was in our computational code. Of course, I can't see the PC of the thread when it hangs because GetContext won't give it to me, so I can't directly confirm that the problem only happens when executing system calls. But, all our system calls are channeled through one point, and so far the evidence is that point was executed when we get the hang. So the indirect evidence suggests GetContext on a thread only fails if a system call is being executed by that thread. I haven't had the energy to build a critical experiment to test this hypothesis yet.

Let me quote from Richter/Nassare's "Windows via C++ 5Ed" which may shed some light:

DWORD SuspendThread(HANDLE hThread);

Any thread can call this function to suspend another thread (as long as you have the thread's handle). It goes without saying (but I'll say it anyway) that a thread can suspend itself but cannot resume itself. Like ResumeThread, SuspendThread returns the thread's previous suspend count. A thread can be suspended as many as MAXIMUM_SUSPEND_COUNT times (defined as 127 in WinNT.h). Note that SuspendThread is asynchronous with respect to kernel-mode execution, but user-mode execution does not occur until the thread is resumed.

In real life, an application must be careful when it calls SuspendThread because you have no idea what the thread might be doing when you attempt to suspend it. If the thread is attempting to allocate memory from a heap, for example, the thread will have a lock on the heap. As other threads attempt to access the heap, their execution will be halted until the first thread is resumed. SuspendThread is safe only if you know exactly what the target thread is (or might be doing) and you take extreme measures to avoid problems or deadlocks caused by suspending the thread.

...

Windows actually lets you look inside a thread's kernel object and grab its current set of CPU registers. To do this, you simply call GetThreadContext:

BOOL GetThreadContext( HANDLE hThread, PCONTEXT pContext);

To call this function, just allocate a CONTEXT structure, initialize some flags (the structure's ContextFlags member) indicating which registers you want to get back, and pass the address of the structure to GetThreadContext. The function then fills in the members you've requested.

You should call SuspendThread before calling GetThreadContext; otherwise, the thread might be scheduled and the thread's context might be different from what you get back. A thread actually has two contexts: user mode and kernel mode. GetThreadContext can return only the user-mode context of a thread. If you call SuspendThread to stop a thread but that thread is currently executing in kernel mode, its user-mode context is stable even though SuspendThread hasn't actually suspended the thread yet. But the thread cannot execute any more user-mode code until it is resumed, so you can safely consider the thread suspended and GetThreadContext will work.

My guess is that GetThreadContext may fail if you just called SuspendThread, while the thread is in kernel mode, and the kernel is locking the thread context block at this time.

Maybe on multicore systems, one core is handling the kernel-mode execution of the thread that it's user mode was just suspended, keep locking the CONTEXT structure of the thread, exactly when the other core is calling GetThreadContext.

Since this behaviour is not documented, I suggest contacting microsoft.

I recently was made aware of this thing called IOCP on windows and i began searching for more information on it but i couldn't find anything up to date (most of the examples were on codeproject almost 5 years old) and not too many guides or tutorials. Can anyone recommend any up to date resources about it in the form of online tutorials or example projects (that you wrote and can share or other open source projects) or even a book about it because if it's as good as it sounds i plan to use it extensively so i will invest in it.

Thank You.

IOCP is a feature that has been in Windows since the dark ages and has changed little in years since. As such, any samples etc. from 5+ years ago should still work pretty well today.

MSDN has some documentation on IOCP: http://msdn.microsoft.com/en-us/library/aa365198%28v=VS.85%29.aspx

Mark Russinovich also wrote up a great intro into IOCP: http://sysinternals.d4rk4.ru/Information/IoCompletionPorts.html

Mark also wrote a more thorough description of Windows' IO infrastructure in "Windows Internals" which is essential reading.

I also strongly recommend Jeffery Richter's "Windows via C/C++" which is also essential reading for anyone embarking on lower-level Windows programming.

HTH.

If you're looking at IOCP from a Network programming point of view then you probably also want to add Network Programming for Microsoft Windows to your list of resources.

There were lots of basic IOCP tutorials on CodeProject back in 2002 when I wrote my articles on IOCP there, so I took a slightly different approach and wrote some code that was, hopefully, reusable as a simple networking framework. This has since grown into a product that I sell. The latest version of the code that's associated with the original CodeProject articles can be found here: http://www.serverframework.com/products---the-free-framework.html I've changed it considerably over the years but the original code still works fine and provides good scalability and is, perhaps, useful as a working example to learn from.

is there any good book/video/web tutorials for multithreading in win32? I only found Addison-Wesley - Multithreading Applications in Win32 (1996) book and might going to order it soon.

Well, you can easily look up a lot of reference material and examples on MSDN in addition to a plethora of books on the subject:

This question might have been partially answered previously also - have a look here - "Recommended Multithreading Book" (but not for Windows specifically)

I think the best source for Win32 programming is Jeffrey Richter's "Windows via C/C++"

Can you suggest a good book for learning Windows Services programming using C++?

Thanks,

Jeffrey Richter's book Windows via C/C++ is a good one. It goes over way more than just Windows Services though.

If you have Visual Studio Pro (or Visual C++ Express), you can dig into the ATL classes around CAtlServiceModuleT. While it is mostly about exposing DCOM interfaces, I've used those classes to write stand-alone Win32 services.

There is also this (old) article on MSDN.

Specifically for Win32 services and related topics such as threads, processes, etc., I very much recommmend: Win32 System Services: The Heart of Windows 98 and Windows 2000 (3rd Edition), by Marshall Brain and Ron Reeves.

The first edition of this book was my introduction to Win32 services-related topics. Marshall Brain wrote that edition in 1993, around the time Windows NT 3.1 was released -- this was the first implementation of the Win32 APIs and the first Microsoft operating system to have Win32 services.

Information at Amazon.com

Does the stack reserved/commited for a thread get freed when

  • the thread terminates
  • the thread object is destroyed
    (i.e. the thread is terminated and all handles to the thread are closed)

?

More broadly, are there significant resources associated with a thread that has terminated, but still exists since there are valid handles to it?


Reason: I need to modify a kind of "scoped singleton", so it doesn't return a single object, but a per-thread object. I cannot rely on thread creation/termination notices, much less on process-wide ones.

At the moment, I store the objects in a map<ThreadID, Object>, with a cache cleanup policy that's suitable for my application. To protect myself from the OS "recycling" thread ID's, I keep an handle to the thread open. (Rec

A side effect would be holding open handles to long-terminated threads in some corner cases.

According to "Windows VIA C/C++" by Richter and Nasarre (A must-have book for any C++ Windwos programmer) p.154:

Terminating a Thread

A thread can be terminated in four ways:

  • The thread function returns. (This is highly recommended.)
  • The thread kills itself by calling the ExitThread function. (Avoid this method.)
  • A thread in the same process or in another one calls the TerminateThread function. (Avoid this method.)
  • The process containing the thread terminates. (Avoid this method.)

The Thread Function Returns

You should always design your thread functions so that they return when you want the thread to terminate. This is the only way to guarantee that all your thread's resources are cleaned up properly.

Having your thread function return ensures the following:

  • All C++ objects created in your thread function will be destroyed properly via their destructors.
  • The operating system will properly free the memory used by the thread's stack.
  • The system will set the thread's exit code (maintained in the thread's kernel object) to your thread function's return value.
  • The system will decrement the usage count of the thread's kernel object.

The ExitThread Function

You can force your thread to terminate by having it call ExitThread:

VOID ExitThread(DWORD dwExitCode);

This function terminates the thread and causes the operating system to clean up all the operating system resources that were used by the thread. However, your C/C++ resources (such as C++ class objects) will not be destroyed. For this reason, it is much better to simply return from your thread function instead of calling ExitThread yourself.

Of course, you use ExitThread's dwExitCode parameter to tell the system what to set the thread's exit code to. The ExitThread function does not return a value because the thread has terminated and cannot execute any more code.

Note The recommended way to have a thread terminate is by having its thread function simply return (as described in the previous section). However, if you use the method described in this section, be aware that the ExitThread function is the Windows function that kills a thread. If you are writing C/C++ code, you should never call ExitThread. Instead, you should use the C++ run-time library function _endthreadex. If you do not use Microsoft's C++ compiler, your compiler vendor will have its own alternative to ExitThread. Whatever this alternative is, you must use it.

The TerminateThread Function

A call to TerminateThread also kills a thread:

BOOL TerminateThread( HANDLE hThread, DWORD dwExitCode);

Unlike ExitThread, which always kills the calling thread, TerminateThread can kill any thread. The hThread parameter identifies the handle of the thread to be terminated. When the thread terminates, its exit code becomes the value you passed as the dwExitCode parameter. Also, the thread's kernel object has its usage count decremented.

Note The TerminateThread function is asynchronous. That is, it tells the system that you want the thread to terminate but the thread is not guaranteed to be killed by the time the function returns. If you need to know for sure that the thread has terminated, you might want to call WaitForSingleObject or a similar function, passing the handle of the thread.

A well-designed application never uses this function because the thread being terminated receives no notification that it is dying. The thread cannot clean up properly, and it cannot prevent itself from being killed.

Note When a thread dies by returning or calling ExitThread, the stack for the thread is destroyed. However, if TerminateThread is used, the system does not destroy the thread's stack until the process that owned the thread terminates. Microsoft purposely implemented TerminateThread in this way. If other still-executing threads were to reference values on the forcibly killed thread's stack, these other threads would raise access violations. By leaving the killed thread's stack in memory, other threads can continue to execute just fine.

In addition, dynamic-link libraries (DLLs) usually receive notifications when a thread is terminating. If a thread is forcibly killed with TerminateThread, however, the DLLs do not receive this notification, which can prevent proper cleanup.

When a Thread Terminates

The following actions occur when a thread terminates:

All User object handles owned by the thread are freed. In Windows, most objects are owned by the process containing the thread that creates the objects. However, a thread owns two User objects: windows and hooks. When a thread dies, the system automatically destroys any windows and uninstalls any hooks that were created or installed by the thread. Other objects are destroyed only when the owning process terminates.

The thread's exit code changes from STILL_ACTIVE to the code passed to ExitThread or TerminateThread.

The state of the thread kernel object becomes signaled.

If the thread is the last active thread in the process, the system considers the process terminated as well.

The thread kernel object's usage count is decremented by 1.

When a thread terminates, its associated thread kernel object doesn't automatically become freed until all the outstanding references to the object are closed.

Once a thread is no longer running, there isn't much any other thread in the system can do with the thread's handle. However, these other threads can call GetExitCodeThread to check whether the thread identified by hThread has terminated and, if it has, determine its exit code:

BOOL GetExitCodeThread( HANDLE hThread, PDWORD pdwExitCode);

The exit code value is returned in the DWORD pointed to by pdwExitCode. If the thread hasn't terminated when GetExitCodeThread is called, the function fills the DWORD with the STILL_ACTIVE identifier (defined as 0x103). If the function is successful, TRUE is returned.

1.How to understand asynchronous io in Windows??

2.If I write/read something to the file using asynchronous io :

WriteFile();
ReadFile();
WriteFile();

How many threads does the OS generate to accomplish these task?

Do the 3 task run simultaneously and in multi-threading way or run one after another just with different order?

3.Can I use multithreading and in each thread using a asynchronous io to read or write the same file?

To your questions:

How many threads does the OS generate to accomplish these task?

Depends if you are using the windows pools, iocp, etc. Generally you decide.

Do the 3 task run simultaneously and in multi-threading way or run one after another just with different order?

This depends on your architecture. On a single-cored machine, the 3 tasks would run one after another and the order would be os decided. On a multi-cored machine these might run together, depending on how the OS scheduled the threads.

3.Can I use multithreading and in each thread using a asynchronous io to read or write the same file?

That is out of my knowledge so someone else would need to answer that one.

I suggest getting a copy of Windows via C/C++ as that has a very large chapter on Asynchronous IO.

I am a reasonably competent C# developer who has experience in threading, web development, windows client development, and basic (recent) experience in multi-platform C. I realize that there are questions out there that cover the insanely wide spectrum of C++.

Given that the C++ and Windows C++ realm is vast with a lot of legacy surface, what I am looking for are books, links, and media that provide a decent crossover point that discuss modern techniques, libraries, and Windows APIs for a seasoned dev (e.g., in what projects would I choose to use the MFC library?).

Update

Additional books that look interesting:

Windows System Programming (4th Edition)

The Old New Thing: Practical Development Throughout the Evolution of Windows

Programming Windows

Update 2

Looks like there is a good channel 9 video series that may become a good source of information as well.

Going Native (Blog)

Channel 9 Video

Update 3

C# to C++ - A Somewhat Short Guide by Michael B. McLaughlin

I'm reading Windows via c/c++. And I just wonder a large file can be mapped to memory.

When we execute an application, a PE file is mapped their process address(user partition).

In 32bit Windows, a Large file(larger than 2GB) can be loaded to user partition? or it will be failed?
If it is possible, Does Paging file help the loading?

You will not be able to do this on 32 bit Windows. Any program running that wants to do this will have at minimum 3 modules loaded:

  • ntdll.dll
  • kernel32.dll
  • yourApplication.exe

yourApplication.exe will by default load at 0x00400000.exe, although you can change that address. ntdll.dll and kernel32.dll will load at their usual load addresses in the high 0x7Dxx0000 range.

On Vista and later operating systems the above paragraph isn't true as the load addresses are randomised, but the dlls will still be present at their own address in the first 2GB of memory.

Thus you will not have enough contiguous space anywhere to load your exceptionally large PE file, even if you start Windows with the /3GB switch.

If you want a good idea of where the various DLLs will load, use VMValidator (free) to visualize the address space of your app. The Virtual Memory View shows you a graphical representation (1 pixel per 4Kb page) of memory. The Pages and Paragraph views show you the actual Virtual Memory status of each memory page and memory paragraph.

For 64 bit Windows, you may be able to load a 2GB 64 bit PE file - there certainly should be enopugh contiguous space to allow it to load in memory. As for whether it would then work I cannot say, you will have to test it.

I want to develop my own GUIs in C++. I mean from the scratch. I am not interested in using the Win32 API, MFC, .NET or anything like that. I want to create and develop everything from scratch. I want to do everything by myself. I just want some links, references or books. Anything that give me guidelines about this.

Can anyone Help?

The Win32 API is the base-API for user-land (i.e, non-kernel-level) programming on the Windows operating system ... you can't do ANYTHING on Windows without going through Win32. So any software that does any operation on Windows will at some point have to make calls to the Win32 API, which in-turn make syscalls to the Windows kernel ... Thus, if you're really interested in programming your own GUI's via raw Win32 calls, I'd suggest getting very familiar with Win32. A good book would be something like "Windows via C/C++" ... it won't give you specific GUI information, but it will give you an in-depth understanding of Win32 that you can use along with the MSDN information to build GUI's from. For actually creating the GUI itself, you can reference the MSDN GDI documentation.

I have started working with C++ recently and am not comfortable with Visual Studio Development Environment and also I do not have proper understanding of MFC, Win32, ATL, COM Terminologies.

From example point of view, I had taken a simple C++ program to see how it works with Visual Studio Environment and I was having some issues to get that code up and running.

I would like to request if someone could point me to some online resources/books where I can get more understanding about Visual Studio Development Environment from C++ perspective and get some knowledge about MFC, Win32, ATL, COM Terminologies than it would be really very helpful to me.

Note: I have checked MSDN library and some related Microsoft sites but when I see HOW DO I kind of video tutorials they are more from .Net/C#/ASP.Net perspective but I am looking for some online resource for C++/VC++ perspective.

The classic book about Win32 is presumably Petzold's. Petzold's book is I think (I've never read it) mostly about GUI programming; whereas the other classic/recommended Win32 book, which is Richter's, is about 'system' (non-GUI) programming.

For learning COM, perhaps Essential COM? Some reviewers praise it, but some others reviews say things like "not for beginners"; but it's how I learned COM, and I found it thorough, low-level, and detailed. It assumes you know C++ (not COM) already.

IMO you don't need books about MFC if you already know C++ and the Win32 API, in which case the reference libraries are sufficient. Alternatively, some people recommend an MFC book like Prosise's.

I want to suspend or save a state of Windows Wista with all opened applications and close them (to have be reopened again later), and start a game (from Steam for example). And when I will quit a game, I want to resume (resume saved state, or resume from suspend) all opened applications to resume work where I left it. I have found this relevant resources:

Can you point me advice on how to create such a feature? I'm just starting with C++, where to look first for a start, and nothing more, just a starting point with resources. To create such a program for Windows Vista x64 Ultimate, I have learned pretty much about C++.

More about the same: Can I make to "Hibernate" only active programs, and leave windows OS active? Is this just need to launch another explorer.exe?

This sounds like a difficult project for a beginner. You'll have to learn quite a bit about the internals of the operating system: process control and how to manipulate virtual memory. A few books you might start with are Windows via C/C++, Windows System Programming, and Windows Internals. A new edition of Windows Internals is coming out in March.