This came up as a question on NTDEV today. After I thought about it for a bit I realized it was actually a pretty good question. You've started writing your driver, or at least thinking about your driver. You know you'll get some data back from the device and you need to get it to the app. How do you do this?
For simplicity's sake, let's say there are two models for a driver to operate in. The "push" model and the "pull" model (of course there are more, but this captures the big ones.) When I say "push" and "pull" here I'm talking about both how the driver decides it should talk to the device, and how it returns the data to the driver.
But before I dive in, realize that I'm talking here about how the driver operates in "steady state" - the time between PNP and Power Management operations, between recovering from device errors/resets, the time when it's just servicing I/O requests.
Pull Model
We'll start with the Pull model because it's easier to understand. In steady state the typical pull-model driver does nothing on its own. When the app wants to talk to the device it sends a request to the driver, which triggers a driver callback. In the simplest case the driver formats a command to send to the device and uses the buffers from the application to hold any data the command returns. When the command returns the app's buffers contain the necessary data. The driver sets the information property on the request to indicate how many bytes of data the app is getting back then completes the request. When the app sees the request completion the data your driver provided will be in its buffers.
It's rarely this simple. Often you'll need to do apply some transformation to the data before you can return it. In that case your driver would allocate a buffer to use in the command you send to the device. When the command completes you use its result to write the correct data into the buffers on the app's requests. Then set information, complete the app's request, and free the intermediate buffer. Better yet, you can make the WDF memory object for the intermediate buffer a child of the request so that it's freed automatically when you complete the request.
Push Model
Say your device is going to bubble data up to your driver when something happens. It might be GPS coordinates, printer status, a finger print scan, etc… How do you get this data back to an application? Clearly you don't want the application to poll for data. All you want to do is push the data you're getting from the device up to the application. How do you do this?
The first thing to understand is that you can only give the application data that it asks for*. Unless the app makes a request and gives you a buffer you don't have a way to put data in its address space. So clearly the Push model is just a variant of the pull model.
In the pull model you let the app send you one or more requests asking for data. You queue those requests for later processing. In the background you have some code which is reading data from the device. When those read commands complete you pickup an app request off the queue, copy your command data into the app's request buffers then complete the app request.
You may choose to copy the data into multiple requests, you may choose not to complete the request if you don't have enough data to fill its buffer with just one command. And if you don't have any pending requests you need to decide whether to just throw the data away or whether to hold it in an internal buffer until the app comes down to ask for more.
The key difference between the push and pull models is whether the driver is sending commands to the device on its own, or whether every device command comes because the app sent your driver a request.
As I said before - it's never this simple. Looking at the UMDF Fx2 sample you see that it uses the pull model for read/write operations on the device and for most of the i/o controls that it supports. However it uses the push model to handle requests to read the state of the DIP switches.
-p
* You can use PNP device events if you need to push the fact that something has happened to any app which might be listening. But it's an unreliable mechanism and isn't suitable as a way to stream data.
To quote from Rev. Lovejoy:
Shoo, short answer: "Yes" with an "If", long answer "No" - with a "But.
There are two types of driver signing, which complicates the answer. As usual any answers I provide are probably vague and can be overridden by standard documentation. In this case the documentation seems to be available on MSDN.
Driver Package Signing:
First there's the standard signing of the 'driver package' (all the files needed to install the driver) which has been around since Win2k. Signing your driver package allows you to avoid the "unsigned driver" popup, and allows end-users to determine whether your package has been tampered with. Personally I think it's goodness, but having never had to go through the signing process I realize it's easy for me to feel that way.
To sign your driver package your INF should have a CatalogFile directive in the [Version] section. The catalog file contains hashes for all the files in your driver package and is then signed to ensure it can't be tampered with.
Code Integrity Signing:
In Vista there's a new code-signing requirement. Kernel-mode binaries that load in 64-bit Windows Vista (or drivers on 32-bit & 64-bit windows that stream protected content) need to be code signed so that the OS can determine whether they have been tampered with.
From a quick scan of the documentation it sounds like this mostly affects the type of certificate you can use when you do your signing. The Code Integrity system (which checks your driver signatures when the driver is loaded) requires something called a "Software Publisher Certificate (SPC)", while regular driver signing can use either an SPC or a "commercial release certificate". However I believe that if you are installing a kernel-mode boot driver you have to make sure your binaries are self signed (signature embedded in the binary) while if you are installing non-boot kernel-mode drivers that you can use a catalog signature (signature embedded in the .cat file installed with the package) instead.
Regardless your driver package will still require a signature in order to get through installation.
So what does all this mean for UMDF?
UMDF drivers do not require Code Integrity Signing since they aren't kernel-mode code. However your driver package still needs to be signed so that it can't be tampered with & can be traced back to the author (or you continue to have unsigned driver popups). And if your package includes any non-Microsoft kernel-mode drivers in addition to the user-mode driver then the Code Integrity Signing rules apply to that too.
To finish with the quote from Rev. Lovejoy:
Uh, if you need additional solace, by the way, I've got a copy of something or other by Art Linkletter in my office.
-p
UMDF's content in the WDK contains a handful of examples for controlling a USB device. Unfortunately they only show how to do it on Vista. The problem was that WinUSB's co-installer wasn't added to the WDK until very late - too late for us to be able to change our samples to show how to use it.
WINUSB introduces a number of new things into the INF because of the complexity of installing it on XP and making it work with the in-box Vista support. In Vista WinUSB is present on the disk but the driver & service aren't "installed" until some INF copies them into place. The WinUSB folks provided an in-box INF (winusb.inf) that you can reference in your INF with the Includes/Needs directives so that you don't have to setup the service entries yourself.
On XP WinUSB doesn't exist until someone installs it. The preferred installation route is to use the WinUsbCoinstaller. However because WINUSB.INF isn't already on the machine in XP, you can't depend on it to setup the WinUSB service entries. And we couldn't rely on the WinUsbCoinstaller to do it because the device installer requires that your INF have an AddService directive for the driver which will act as the FDO.
The result is that installing WinUSB requires you to add a few things to your INF. Some of these are already demonstrated in the existing sample INFs. Others are new, and there's one "bug fix". I've added the full INF as an attachment to this blog entry.
So starting with the Fx2 driver's INF, here are the necessary changes to build a driver package that will install both on XP and Vista.
-
Include sections from WINUSB.INF in your INF
Specifically in your [DDInstall.NT] section you add:
Include=WINUSB.INF ; Import sections from WINUSB.INF
Needs=WINUSB.NT ; Run the CopyFiles & AddReg directives
The first tells Device Installation to locate WINUSB.INF and read in all of its sections. The second tells DI that it should import any directives in the "WINUSB.NT" section into this section as well. WINUSB.NT contains CopyFiles & AddReg directives that tell Vista to copy the WinUSB.Sys binary as part of your driver package & sets up some registry entries.
Now you may recall I said that there was a problem on XP because this INF doesn't exist. However when you ask DI to include an INF file that doesn't exist (or say you Need a section that doesn't exist) the installer just ignores you. Normally that would drive me nuts (I hate hidden errors) but it works in my favor in this case.
On XP these two directives are benign and we depend on the WinUsbCoinstaller to copy the files into place and setup the registry entries. On Vista WinUsbCoinstaller doesn't do anything and we depend on these directives to copy the binaries.
This leads us to step 2 ...
-
Copy & Invoke the WinUsbCoinstaller in your INF
To invoke the WinUSB coinstaller requires three things in your INF:
-
Add the file to [SourceDisksFiles] so that DI knows where to get the file from
[SourceDisksFiles]
...
WinUsbCoinstaller.dll=1
-
Add the file to [CoInstallers_CopyFiles] so that DI knows that it needs to copy it
CoInstallers_CopyFiles]
...
WinUsbCoinstaller.dll
-
Add the name of the coinstaller to the CoInstallers32 registry entry so that DI knows to invoke it.
[CoInstallers_AddReg]
HKR,,CoInstallers32,0x00010000,"WudfUpdate_$UMDFCOINSTALLERVERSION$.dll",\
"WinUsbCoinstaller.dll", \
"WdfCoInstaller01005.dll,WdfCoInstaller"
Note that the names of the last two sections need to match what you provide to CopyFiles and AddReg in your INF's [DDInstall.Coinstallers] section.
-
Fix the bug in the [WinUsb_Install] section
WinUsb requires version 1.5 of KMDF, not version 1.0. We didn't catch this before the samples shipped. Change the section to look like:
[WinUsb_Install]
KmdfLibraryVersion=1.5
-
Include WinUsbCoinstaller.dll as part of your signed driver package
Along with WudfUpdate_01005.dll & WdfCoinstaller01005.dll you also now need to copy WinUsbCoinstaller.dll & add it to the catalog for your INF. You can find this file in c:\winddk\6000\redist\winusb\<arch>.
Like the KMDF and UMDF coinstallers, it comes in a _chk version also. If you're installing your driver (whether you built it checked or free) on a checked system you should use the _chk version of the DLL.
Four little changes and you're done. The same changes apply to any of the UMDF USB samples.
-p
I finally got back to my blog after coming back from vacation (two week drive down to disneyland with the wife … lots of fun) and noticed that the CSS overrides I'd done looked even worse than before.
However they also loaded a lot of new skins into MSDN blogs, so I picked a different one.
Expect the look to be in flux while I find one that I like.
-p
I got looped into a discussion about the WDK glossary entry for "Windows Driver Model". It got me thinking about how difficult it is to explain things in the driver space to non-driver folks.
When I was working in storage I used to have this problem all the time. Someone, usually a family member, would ask me what I did and I'd have to talk about what a "storage device" was, about what miniports were, etc… For a while I gave up and described my job as "something technical and utterly uninteresting". Finally I settled on "I make the light on your hard-disk blink".
Then I moved to the WDF team and everything became complicated again. Explaining what WDF is to a non-driver person is a bit like explaining what colors are to someone who is color blind.
So here's my attempt to explain WDM to someone sort of technical who has no idea what a device driver does:
Windows Driver Model
An architecture for Windows extensions (device drivers) which allow the operating system core to communicate with a wide variety of devices. WDM defines a Device Driver Interface (DDI) for drivers to interact with each other and the OS, and a set of required behaviors that device drivers must implement in order to work well with the rest of the system. WDM and WDM drivers work together to provide device I/O, Plug and Play, Power Management and WMI support.
WDM is a packet-based I/O architecture. Applications (as well as the OS and other drivers) can make requests of particular devices. WDM represents these requests as I/O Request Packets (IRPs) and sends them to a particular set of device drivers for processing. The drivers typically work in a layered fashion – each performing some portion of the request then handing the request off to the next driver – until the bottom most driver programs the device to perform the requested operation(s).
Not all devices are supported by WDM drivers. Some device classes (such as storage devices or network adapters) provide their own class-specific DDIs and behaviors which represent a subset of WDM. Other drivers may be implemented using the Windows Driver Frameworks, which provides a simpler general-purpose infrastructure for writing drivers.
The question came up on the NTDEV mailing list a while back about how the "Scheduler" works in Windows … specifically about what thread it ran on and how it got control.
The answer is that you should ignore this idea that there's a "scheduler" as an independent entity. It doesn't exist like that in Windows.
In windows there are basically two functions for scheduling in the dispatcher. The first selects the most appropriate thread to run on a processor at any given time based on a number of factors such as thread priority, thread affinity, ideal processor, etc… . The second initiates a context switch between the current thread and the new one.
The Ke subsystem implements these functions and uses them to schedule threads in a distributed manner. Basically any time a Ke function is called which changes the state of a thread (to block it, to wake it up, to change its priority, to mark it as having exceeded a quantum) that function will invoke these two other functions to see if a different thread should be running and, if so, to swap over to it.
So you've got some user-mode code on a processor. That code will continue to run until it:
- Calls WaitForSingleObject, which calls the NtWaitForSingleObject system call which transitions to kernel mode and (eventually) calls KeWaitForSingleObject. KeWaitForSingleObject links the current thread onto the wait list for the object (say an event) then selects another thread which could be run and does a context switch
- Is interrupted by a device interrupt which completes an I/O request initiated by another process. Often this involves boosting the priority of the other process by calling a Ke function. This function sets the new priority, then calls the selection function to see what process should be running on the processor. If there's a better candidate after the priority boost then Ke will swap to the new thread
There are a number of other conditions which can cause a context switch, and I've dramatically oversimplified some of them, but hopefully this will help you get the idea. The responsibility for scheduling in NT is distributed across all threads in the system, which eventually end up in a Ke routine.
Nar's talk on the NT I/O manager is now up on channel 9.
Ilias is the latest addition to the UMDF development team. I'm really excited that he's decided to start blogging about his initial experience learning about how WDM and UMDF drivers work. Hopefully this can serve as a helpful introduction for anyone who is just starting off with driver development.
It appears my Channel 9 interview is live. I haven't actually watched it yet, so I might regret putting this link up before I do, but people are going to find it anyway.
Hopefully I sound insightful. And hopefully the squeaky voice is due to an audio driver bug on my Vista machine or I might have to stop speaking in public J.
Enjoy,
-p
UMDF defines all of its interfaces using COM interfaces, rather than the flat C-style DDI that the KMDF team used. When kernel developers see this their first question is often "why did you do that?" and it's an understandable question. I had a long discussion at WinHEC about this exact topic.
First not everyone is sold on the flat C DDI that we put together for KMDF. In the polls we've done, the sample group (let's be optimistic and call it the "industry") is split about evenly on whether they want C or C++. However for folks that concentrate more on user-mode stuff there's a greater preference for C++. So we wanted a DDI that would work well with C++ but that wouldn't force you to use it.
Using straight C++ objects also wouldn't work. C++ has its nice points, but sharing objects across DLL boundaries isn't one of them. There's no way to publish only your "public" members, there are concerns about whether our compiler will do the same name mangling from release to release with exports (long story), there's the fragile base class problem, there are all sorts of issues with the new & delete operators, and it can't be used from C without a wrapper library.
We could have used C++ abstract classes – describing the interfaces that our objects exposed without any of the pesky internal details. But we'd then have to add reference counting on top of that, and some method of casting one interface type to another, and when you're all done it looks like every object supports IUnknown.
So the bottom line is that we chose COM because we had to expose something that acted an awful lot like COM. Rather than create our own semantic which everyone would have to learn, we decided to borrow from an existing one that's already understood by many developers.
We've worked very hard to keep the COM runtime goop out of UMDF. UMDF drivers don't register as COM DLLs, they don't get to do any of the weird COM synchronization stuff, and they don't have to worry about IDispatch or monikers or whether they're running in-proc/out-of-proc. We try to use the COM programming model but without the weird magic that the COM runtime introduces. We do allow you to use something like ATL though if you're tolerant of a little magic and want to make the COM boilerplate easier. And I have to admit that as much as I dislike magical templated macros, being able to use CComPtr to do my reference counting for me is a helpful thing.
-p
Trying to get the UMDF Beta 2 aligned for Windows Media Player & trying to get the Vista RC out the door have been pretty time consuming and my blog muse decided to take a nice long nap. Now that my team is more or less "done" for Vista I'm hopeful that I'll have more time to work on other things, including my blog.
I'm sorry it's taken me so long to write about this. The
last week or two have been pretty hectic as we try to lockdown for Vista.
But we still want people to download the UMDF 1.0 beta, and here are the
instructions for that.
The first step, of course, is to install the WDK. You
can either use the Beta 2 WDK or the 5456 version which was recently published
on the beta site. If you don't have either of these installed yet, you can
my earlier blog entry on
installing the WDK.
Next you need to download the UMDF beta bits and install those
on your development machine as well. Go to the connect.microsoft.com site
for the beta & look under downloads. You will see several items but you
want to download and run either "UMDFx86" or "UMDF64" depending on the
architecture of your development machine (not your test machine).
Download and install the MSI and you'll be ready to go.
To actually build one of the samples you'll want to start a
"Windows Vista and Windows Server Longhorn" build environment from the WDK.
This is burried under "Windows Driver Kits" on your start menu. Be sure
you either use the 5483 or 5456 WDK ... if you have Beta 1 or the February CTP
kit still installed those may not work.
Once you've started your build window, you need to run a
script to tell it to use the headers & all from the MSI you just installed
rather than the ones in the kit. Do the following:
cd \WinDDK\WDF\UMDF10_xp
set_umdf_env.cmd
cd src\umdf
build -cZ
And you've just built the UMDF 1.0 samples.
Next time i'll walk through installing the echo sample driver.
YIPPY!!!
Sometimes it seems like releasing software is more complicated than developing it. We really wanted to have this at WinHEC, but that just wasn't going to happen. Fortunately I have an amazing team to help push through all the problems and process imposed upon us. So we have a beta release.
If you're already signed up for the Windows Driver Kit beta then you should have email telling you that the UMDF beta is available. The mail helpfully fails to tell you where to get the beta - it's on the download page for the WDK beta.
If you aren't signed up but you've read this far you clearly want to. Go ahead and do it. Unless you're chicken* that is (or don't have the bandwidth to download a 2+ GB kit).
I really want to write some more about using it, but my team is trying to lockdown for the Vista RC and I don't have time today. In the next few days i'll walk through the steps of installing it, compiling the samples and getting the resulting driver installed.
Until then, the readme file is your friend.
Enjoy.
-p
* they say goading can be an effective motivator. Let's try it now.
When i first saw the new start menu I'll admit that i thought it was overkill. Who in the world wants to search from the start menu after all.
But i'm finding that it's actually pretty useful. In the days of yore when i wanted to launch Excel or the WDK documentation i had to dig through this long list of folders to find what i wanted. Is Excel under the "Office" or "Microsoft Office" sub-directory in the start menu? I know the WDK documentation has WDK in the name, but where.
The new start menu actually makes this much easier. Hit the Windows key and type in "WDK" (after you've installed the WDK beta on your Vista machine) or "excel". It searches through the contents of the start menu and lists all the items that contain that word. Usually the executable with that name is first.
It can be a little slow, but i have to admit it's actually causing me to use the start menu more rather than learning the real exe name and starting the program from the Run dialog.
I got back from vacation today and went to
connect.microsoft.com to see
if the WDK beta was up yet, and i'm happy to say that it is. It's a fairly
large (2.3 GB) ISO image to download and i'm pulling it down now to make
sure it works okay.
Here are the steps i had to follow to get into the beta:
-
Navigate to connect.microsoft.com
and sign in using your passport account (the passport link is at
the top right of the web page).
-
Go to the Available Programs
page (there's a menu link on the left side of the window) and scroll
down to find the Microsoft Windows Driver Kit (WDK) Beta Program entry
then click on the Apply/ link.
-
Decide if you agree with the terms and conditions. If you do then
click on the I Agree button at the bottom of the screen.
-
This should bring you to the Mandatory Surveys page. The
Beta 2 Feedback survey is very, very short so you should just
take it. If you want to skip it there's a link for that too, but I'm
guessing that you'll be forced to take it by 6/15 if you want to keep
access to the beta site (based on the web page content).
-
Either way you'll end up at the web page for the WDK beta. There's
nothing really on the page, but you can get the kit from the
Downloads link in the menu on the left-hand side of the screen.
-
On the downloads page, choose the last item (WDK Beta 2). Downloading
this requires that you install an ActiveX control, and the download
will take a while. Once you're done you should be able to burn it
to a DVD using your favorite tool, or use the
Virtual CD-ROM tool
to mount the image.
from Microsoft.
For purposes of this discussion, we'll assume you mount the ISO image
(either with a DVD or VirtualCD) on z:.
-
Open Z: and run Installer.exe. This will let you install the
prerequisites and then the build environment and doc set. Install
whatever other parts you see fit.
You should now have a kit installed which will let you build UMDF drivers
for use on Windows Vista (beta of XP support is on the way.) I'll talk
about getting started with that later.
Enjoy.
-p