The Complete Idiot's Guide to Writing Shell Extensions - Part I

A step-by-step tutorial on writing shell extensions

Contents

README.TXT

This is the stuff I want you to read first, before proceeding on or posting messages to this article's discussion board.

This series of articles was originally written for VC 6 users. Now that VC 8 is out, I felt it was about time to update the articles to cover VC 7.1. 😉 (Also, the automatic 6-to-7 conversion done by VC 7.1 doesn't always go smoothly so VC 7.1 users could get stuck when trying to use the demo source code.) So as I go through and update this series, the articles will be updated to reflect new VC 7.1 features, and I'll have VC 7.1 projects in the source downloads.

Important note for VC 2005 users: The Express edition of VC 2005 does not come with ATL or MFC. Since the articles in this series use ATL, and some use MFC, you won't be able to use the Express editions with the articles' sample code.

If you are using VC 6, you should get an updated Platform SDK. You can use the web install version, or download the CAB files or an ISO image and run the setup locally. Be sure you use the utility to add the SDK include and lib directories to the VC search path. You can find this in the Visual Studio Registration folder in the Platform SDK program group. It's a good idea to get the latest Platform SDK even if you're using VC 7 or 8 so you have the latest headers and libs.

Important note for VC 7 users: You must make a change to the default include path if you haven't updated your Platform SDK. Make sure that $(VCInstallDir)PlatformSDK\include is first in the list, above ($VCInstallDir)include, as shown here:

Since I haven't used VC 8 yet, I don't know if the sample code will compile on 8. Hopefully the 7-to-8 upgrade process will work better than the 6-to-7 process did. Please post on this article's forum if you have any trouble with VC 8.

Introduction to the Series

A shell extension is a COM object that adds some kind of functionality to the Windows shell (Explorer). There are all kinds of extensions out there, but very little easy-to-follow documentation about what they are. (Although I bet the situation has improved during the six years since I originally wrote that!) I highly recommend Dino Esposito's great book Visual C++ Windows Shell Programming (ISBN 1861001843) if you want an in-depth look into lots of aspects of the shell, but for folks who don't have the book, or only care about shell extensions, I've written up this tutorial that will astound and amaze you, or failing that, get you well on your way to understanding how to write your own extensions. This guide assumes you are familiar with the basics of COM and ATL. If you need a refresher on COM basics, check out my Intro to COM article.

Part I contains a general introduction to shell extensions, and a simple context menu extension to whet your appetite for the following parts.

There are two parts in the term "shell extension." Shell refers to Explorer, and extension refers to code you write that gets run by Explorer when a predetermined event happens (e.g., a right-click on a .DOC file). So a shell extension is a COM object that adds features to Explorer.

A shell extension is an in-process server that implements some interfaces that handle the communication with Explorer. ATL is the easiest way to get an extension up and running quickly, since without it you'd be stuck writing QueryInterface() and AddRef() code over and over. It is also much easier to debug extensions on Windows NT-based OSes, as I will explain later.

There are many types of shell extensions, each type being invoked when different events happen. Here are a few of the more common types, and the situations in which they are invoked:

Type When it's invoked What it does
Context menu handler User right-clicks on a file or folder. In shell versions 4.71+, also invoked on a right-click in the background of a directory window. Adds items to the context menu.
Property sheet handler Properties dialog displayed for a file. Adds pages to the property sheet.
Drag and drop handler User right-drags items and drops them on a directory window or the desktop. Adds items to the context menu.
Drop handler User drags items and drops them on a file. Any desired action.
QueryInfo handler (shell version 4.71+) User hovers the mouse over a file or other shell object like My Computer. Returns a string that Explorer displays in a tooltip.

Introduction to Part I

By now you many be wondering what an extension looks like in Explorer. One example is WinZip - it contains many types of extensions, one of them being a context menu handler. Here some commands that WinZip adds to the context menu for compressed files:

 [WinZip menu items - 4K]

WinZip contains the code that adds the menu items, provides fly-by help (text that appears in Explorer's status bar), and carries out the appropriate actions when the user chooses one of the WinZip commands.

WinZip also contains a drag and drop handler. This type is very similar to a context menu extension, but it is invoked when the user drags a file using the right mouse button. Here is what WinZip's drag and drop handler adds to the context menu:

 [WinZip menu items - 2K]

There are many other types (and Microsoft keeps adding more in each version of Windows!). For now, we'll just look at context menu extensions, since they are pretty simple to write and the results are easy to see (instant gratification!).

Before we begin coding, there are some tips that will make the job easier. When you cause a shell extension to be loaded by Explorer, it will stay in memory for a while, making it impossible to rebuild the DLL. To have Explorer unload extensions more often, create this registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Explorer\AlwaysUnloadDLL

and set the default value to "1". On 9x, that's the best you can do. On NT, go to this key:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer

and create a DWORD called DesktopProcess with a value of 1. This makes the desktop and Taskbar run in one process, and subsequent Explorer windows each run in its own process. This means that you can do your debugging with a single Explorer window, and when you close it, your DLL is automatically unloaded, avoiding any problems with the file being in use. You will need to log off and back on for these changes to take effect.

I will explain how to debug on 9x a little later.

Using AppWizard to Get Started

Let's start simple, and make an extension that just pops up a message box to show that it's working. We'll hook the extension up to .TXT files, so our extension will be called when the user right-clicks a text file.

OK, it's time to get started! What's that? I haven't told you how to use the mysterious shell extension interfaces yet? Don't worry, I'll be explaining as I go along. I find that it's easier to follow along with examples if the concepts are explained, and followed immediately by sample code. I could explain everything first, then get to the code, but I find that harder to absorb. Anyway, fire up VC and we'll get started.

Run the AppWizard and make a new ATL COM program. We'll call it "SimpleExt". Keep all the default settings in the AppWizard, and click Finish. We now have an empty ATL project that will build a DLL, but we need to add our shell extension's COM object. In the ClassView tree, right-click the SimpleExt classes item, and pick New ATL Object. (In VC 7, right-click the item and pick Add|Add Class.)

In the ATL Object Wizard, the first panel already has Simple Object selected, so just click Next. On the second panel, enter "SimpleShlExt" in the Short Name edit box (the other edit boxes on the panel will be filled in automatically):

By default, the wizard creates a COM object that can be used from C and script-based clients through OLE Automation. Our extension will only be used by Explorer, so we can change some settings to remove the Automation features. Go to the Attributes page, and change the Interface type to Custom, and change the Aggregation setting to No:

When you click OK, the wizard creates a class called CSimpleShlExt that contains the basic code for implementing a COM object, and adds this class to the project. We will add our code to this class.

The Initialization Interface

When our shell extension is loaded, Explorer calls our QueryInterface() function to get a pointer to an IShellExtInit interface. This interface has only one method, Initialize(), whose prototype is:

HRESULT IShellExtInit::Initialize (
  LPCITEMIDLIST pidlFolder,
  LPDATAOBJECT pDataObj,
  HKEY hProgID )

Explorer uses this method to give us various information. pidlFolder is the PIDL of the folder containing the files being acted upon. (A PIDL [pointer to an ID list] is a data structure that uniquely identifies any object in the shell, whether it's a file system object or not.) pDataObj is an IDataObject interface pointer through which we retrieve the names of the files being acted upon. hProgID is an open HKEY which we can use to access the registry key containing our DLL's registration data. For this simple extension, we'll only need to use the pDataObj parameter.

To add this to our COM object, open the SimpleShlExt.h file, and add the lines listed below in bold. Some of the COM-related code generated by the wizard isn't needed, since we're not implementing our own interface, so I've indicated the code that can be removed with strikeout type:

#include <shlobj.h>
#include <comdef.h>
 
class ATL_NO_VTABLE CSimpleShlExt :
  public CComObjectRootEx<CComSingleThreadModel>,
  public CComCoClass<CSimpleShlExt, &CLSID_SimpleShlExt>,
  public ISimpleShlExt,
  public IShellExtInit
{
  BEGIN_COM_MAP(CSimpleShlExt)
    COM_INTERFACE_ENTRY(ISimpleShlExt)
    COM_INTERFACE_ENTRY(IShellExtInit)
  END_COM_MAP()

The COM_MAP is how ATL implements QueryInterface(). It tells ATL what interfaces other programs can retrieve from our COM objects.

Inside the class declaration, add the prototype for Initialize(). We'll also need a buffer to hold a filename:

protected:
  TCHAR m_szFile[MAX_PATH];
 
public:
  // IShellExtInit
  STDMETHODIMP Initialize(LPCITEMIDLIST, LPDATAOBJECT, HKEY);

Then, in the SimpleShlExt.cpp file, add the definition of the function:

STDMETHODIMP CSimpleShlExt::Initialize ( 
  LPCITEMIDLIST pidlFolder,
  LPDATAOBJECT pDataObj,
  HKEY hProgID )

What we'll do is get the name of the file that was right-clicked, and show that name in a message box. If there is more than one selected file, you could access them all through the pDataObj interface pointer, but since we're keeping this simple, we'll only look at the first filename.

The filename is stored in the same format as the one used when you drag and drop files on a window with theWS_EX_ACCEPTFILES style. That means we get the filenames using the same API: DragQueryFile(). We'll begin the function by getting a handle to the data contained in the IDataObject:

HRESULT CSimpleShlExt::Initialize(...)
{
FORMATETC fmt = { CF_HDROP, NULL, DVASPECT_CONTENT,
                  -1, TYMED_HGLOBAL };
STGMEDIUM stg = { TYMED_HGLOBAL };
HDROP     hDrop;
 
  // Look for CF_HDROP data in the data object. If there
  // is no such data, return an error back to Explorer.
  if ( FAILED( pDataObj->GetData ( &fmt, &stg ) ))
    return E_INVALIDARG;
 
  // Get a pointer to the actual data.
  hDrop = (HDROP) GlobalLock ( stg.hGlobal );
 
  // Make sure it worked.
  if ( NULL == hDrop )
    return E_INVALIDARG;

Note that it's vitally important to error-check everything, especially pointers. Since our extension runs in Explorer's process space, if our code crashes, we take down Explorer too. On 9x, such a crash might necessitate rebooting the computer.

Now that we have an HDROP handle, we can get the filename we need.

  // Sanity check – make sure there is at least one filename.
UINT uNumFiles = DragQueryFile ( hDrop, 0xFFFFFFFF, NULL, 0 );
HRESULT hr = S_OK;
  
  if ( 0 == uNumFiles )
    {
    GlobalUnlock ( stg.hGlobal );
    ReleaseStgMedium ( &stg );
    return E_INVALIDARG;
    }
 
  // Get the name of the first file and store it in our
  // member variable m_szFile.
  if ( 0 == DragQueryFile ( hDrop, 0, m_szFile, MAX_PATH ) )
    hr = E_INVALIDARG;
 
  GlobalUnlock ( stg.hGlobal );
  ReleaseStgMedium ( &stg );
 
  return hr;
}

If we return E_INVALIDARG, Explorer will not call our extension for this right-click event again. If we return S_OK, then Explorer will call QueryInterface() again and get a pointer to another interface: IContextMenu.

The Interface for Interacting with the Context Menu

Once Explorer has initialized our extension, it will call the IContextMenu methods to let us add menu items, provide fly-by help, and carry out the user's selection.

Adding IContextMenu to our shell extension is similar to adding IShellExtInit. Open up SimpleShlExt.h and add the lines listed here in bold:

class ATL_NO_VTABLE CSimpleShlExt : 
    public CComObjectRootEx<CComSingleThreadModel>,
    public CComCoClass<CSimpleShlExt, &CLSID_SimpleShlExt>,
    public IShellExtInit,
    public IContextMenu
{
  BEGIN_COM_MAP(CSimpleShlExt)
    COM_INTERFACE_ENTRY(IShellExtInit)
    COM_INTERFACE_ENTRY(IContextMenu)
  END_COM_MAP()

And then add the prototypes for the IContextMenu methods:

public:
  // IContextMenu
  STDMETHODIMP GetCommandString(UINT, UINT, UINT*, LPSTR, UINT);
  STDMETHODIMP InvokeCommand(LPCMINVOKECOMMANDINFO);
  STDMETHODIMP QueryContextMenu(HMENU, UINT, UINT, UINT, UINT);

Modifying the context menu

IContextMenu has three methods. The first one, QueryContextMenu(), lets us modify the menu. The prototype of QueryContextMenu() is:

HRESULT IContextMenu::QueryContextMenu (
  HMENU hmenu, UINT uMenuIndex, UINT uidFirstCmd,
  UINT uidLastCmd, UINT uFlags );

hmenu is a handle to the context menu. uMenuIndex is the position in which we should start adding our items. uidFirstCmd and uidLastCmd are the range of command ID values we can use for our menu items. uFlags indicates why Explorer is calling QueryContextMenu(), and I'll get to this later.

The return value is documented differently depending on who you ask. Dino Esposito's book says it's the number of menu items added by QueryContextMenu(). The MSDN docs from VC 6 says it's the command ID of the last menu item we add, plus 1. The online MSDN says this:

If successful, returns an HRESULT value that has its severity value set to SEVERITY_SUCCESS and its code value set to the offset of the largest command identifier that was assigned, plus one. For example, assume that idCmdFirst is set to 5 and you add three items to the menu with command identifiers of 5, 7, and 8. The return value should be MAKE_HRESULT(SEVERITY_SUCCESS, 0, 8 - 5 + 1). Otherwise, it returns an OLE error value.

I've been following Dino's explanation so far in the code I've written, and it's worked fine. Actually, his method of making the return value is equivalent to the online MSDN method, as long as you start numbering your menu items with uidFirstCmd and increment it by 1 for each item.

Our simple extension will have just one item, so the QueryContextMenu() function is quite simple:

HRESULT CSimpleShlExt::QueryContextMenu (
  HMENU hmenu, UINT uMenuIndex, UINT uidFirstCmd,
  UINT uidLastCmd, UINT uFlags )
{
  // If the flags include CMF_DEFAULTONLY then we shouldn't do anything.
  if ( uFlags & CMF_DEFAULTONLY )
    return MAKE_HRESULT ( SEVERITY_SUCCESS, FACILITY_NULL, 0 );
 
  InsertMenu ( hmenu, uMenuIndex, MF_BYPOSITION,
               uidFirstCmd, _T("SimpleShlExt Test Item") );
 
  return MAKE_HRESULT ( SEVERITY_SUCCESS, FACILITY_NULL, 1 );
}

The first thing we do is check uFlags. You can look up the full list of flags in MSDN, but for context menu extensions, only one value is important: CMF_DEFAULTONLY. This flag tells namespace extensions to add only the default menu item. Shell extensions should not add any items when this flag is present. That's why we return 0 immediately if the CMF_DEFAULTONLY flag is present. If that flag isn't present, we modify the menu (using the hmenu handle), and return 1 to tell the shell that we added 1 menu item.

Showing fly-by help in the status bar

The next IContextMenu that can be called is GetCommandString(). If the user right-clicks a text file in an Explorer window, or selects a text file and then clicks the File menu, the status bar will show fly-by help when our menu item is highlighted. Our GetCommandString() function will return the string that we want Explorer to show.

The prototype for GetCommandString() is:

HRESULT IContextMenu::GetCommandString (
  UINT idCmd, UINT uFlags, UINT* pwReserved,
  LPSTR pszName, UINT cchMax );

idCmd is a zero-based counter that indicates which menu item is selected. Since we have just one menu item, idCmd will always be zero. But if we had added, say, 3 menu items, idCmd could be 0, 1, or 2. uFlags is another group of flags that I'll describe later. We can ignore pwReserved. pszName is a pointer to a buffer owned by the shell where we will store the help string to be displayed. cchMax is the size of the buffer. The return value is one of the usual HRESULT constants, such as S_OK or E_FAIL.

GetCommandString() can also be called to retrieve a "verb" for a menu item. A verb is a language-independent string that identifies an action that can be taken on a file. The docs for ShellExecute() have more to say, and the subject of verbs is best suited for another article, but the short version is that verbs can be either listed in the registry (such as "open" and "print"), or created dynamically by context menu extensions. This lets an action implemented in a shell extension be invoked by a call to ShellExecute().

Anyway, the reason I mentioned all that is we have to determine why GetCommandString() is being called. If Explorer wants a fly-by help string, we provide it. If Explorer is asking for a verb, we'll just ignore the request. This is where the uFlags parameter comes into play. If uFlags has the GCS_HELPTEXT bit set, then Explorer is asking for fly-by help. Additionally, if the GCS_UNICODE bit is set, we must return a Unicode string.

The code for our GetCommandString() looks like this:

#include <atlconv.h>  // for ATL string conversion macros
 
HRESULT CSimpleShlExt::GetCommandString (
  UINT idCmd, UINT uFlags, UINT* pwReserved,
  LPSTR pszName, UINT cchMax )
{
USES_CONVERSION;
 
  // Check idCmd, it must be 0 since we have only one menu item.
  if ( 0 != idCmd )
    return E_INVALIDARG;
 
  // If Explorer is asking for a help string, copy our string into the
  // supplied buffer.
  if ( uFlags & GCS_HELPTEXT )
    {
    LPCTSTR szText = _T("This is the simple shell extension's help");
 
    if ( uFlags & GCS_UNICODE )
      {
      // We need to cast pszName to a Unicode string, and then use the
      // Unicode string copy API.
      lstrcpynW ( (LPWSTR) pszName, T2CW(szText), cchMax );
      }
    else
      {
      // Use the ANSI string copy API to return the help string.
      lstrcpynA ( pszName, T2CA(szText), cchMax );
      }
 
    return S_OK;
    }
 
  return E_INVALIDARG;
}

Nothing fancy here; I just have the string hard-coded and convert it to the appropriate character set. If you have never used the ATL conversion macros, check out Nish and my article on string wrapper classes; they make life a lot easier when having to pass Unicode strings to COM methods and OLE functions.

One important thing to note is that the lstrcpyn() API guarantees that the destination string will be null-terminated. This is different from the CRT function strncpy(), which does not add a terminating null if the source string's length is greater than or equal to cchMax. I suggest always using lstrcpyn(), so you don't have to insert checks after every strncpy() call to make sure the strings end up null-terminated.

Carrying out the user's selection

The last method in IContextMenu is InvokeCommand(). This method is called if the user clicks on the menu item we added. The prototype for InvokeCommand() is:

HRESULT IContextMenu::InvokeCommand (
  LPCMINVOKECOMMANDINFO pCmdInfo );

The CMINVOKECOMMANDINFO struct has a ton of info in it, but for our purposes, we only care about lpVerband hwnd. lpVerb performs double duty - it can be either the name of the verb that was invoked, or it can be an index telling us which of our menu items was clicked on. hwnd is the handle of the Explorer window where the user invoked our extension; we can use this window as the parent window for any UI that we show.

Since we have just one menu item, we'll check lpVerb, and if it's zero, we know our menu item was clicked. The simplest thing I could think to do is pop up a message box, so that's just what this code does. The message box shows the filename of the selected file, to prove that it's really working.

HRESULT CSimpleShlExt::InvokeCommand (
  LPCMINVOKECOMMANDINFO pCmdInfo )
{
  // If lpVerb really points to a string, ignore this function call and bail out.
  if ( 0 != HIWORD( pCmdInfo->lpVerb ) )
    return E_INVALIDARG;
 
  // Get the command index - the only valid one is 0.
  switch ( LOWORD( pCmdInfo->lpVerb ) )
    {
    case 0:
      {
      TCHAR szMsg[MAX_PATH + 32];
 
      wsprintf ( szMsg, _T("The selected file was:\n\n%s"), m_szFile );
 
      MessageBox ( pCmdInfo->hwnd, szMsg, _T("SimpleShlExt"),
                   MB_ICONINFORMATION );
 
      return S_OK;
      }
    break;
 
    default:
      return E_INVALIDARG;
    break;
    }
}

Other code details

There are a couple more tweaks we can make to the wizard-generated code to remove OLE Automation features that we don't need. First, we can remove some registry entries from the SimpleShlExt.rgs file (the purpose of this file is explained in the next section):

HKCR
{
  SimpleExt.SimpleShlExt.1 = s 'SimpleShlExt Class'
  {
    CLSID = s '{5E2121EE-0300-11D4-8D3B-444553540000}'
  }
  SimpleExt.SimpleShlExt = s 'SimpleShlExt Class'
  {
    CLSID = s '{5E2121EE-0300-11D4-8D3B-444553540000}'
    CurVer = s 'SimpleExt.SimpleShlExt.1'
  }
  NoRemove CLSID
  {
    ForceRemove {5E2121EE-0300-11D4-8D3B-444553540000} = s 'SimpleShlExt Class'
    {
      ProgID = s 'SimpleExt.SimpleShlExt.1'
      VersionIndependentProgID = s 'SimpleExt.SimpleShlExt'
      InprocServer32 = s '%MODULE%'
      {
        val ThreadingModel = s 'Apartment'
      }
      'TypeLib' = s '{73738B1C-A43E-47F9-98F0-A07032F2C558}'
    }
  }
}

We can also remove the type library from the DLL's resources. Click View|Resource Includes. In the Compile-time directives box, you'll see a line that includes the type library:

Remove that line, then click OK when VC warns about modifying the includes.

In VC 7, the command is in a different location. On the Resource View tab, right-click the SimpleExt.rc folder, and pick Resource Includes on the menu.

Now that we've removed the type library, we need to change two lines of code and tell ATL that it shouldn't do anything with the type library. Open SimpleExt.cpp, go to the DllRegisterServer() function, and change the RegisterServer() parameter to FALSE:

STDAPI DllRegisterServer()
{
//...
  return _Module.RegisterServer(TRUE FALSE);
}

DllUnregisterServer() needs a similar change:

STDAPI DllUnregisterServer()
{
//...
  return _Module.UnregisterServer(TRUE FALSE);
}

Registering the Shell Extension

So now we have all of the COM interfaces implemented. But... how do we get Explorer to use our extension? ATL automatically generates code that registers our DLL as a COM server, but that just lets other apps use our DLL. In order to tell Explorer our extension exists, we need to register it under the key that holds info about text files:

HKEY_CLASSES_ROOT\txtfile

Under that key, a key called ShellEx holds a list of shell extensions that will be invoked on text files. Under ShellEx, the ContextMenuHandlers key holds a list of context menu extensions. Each extension creates a subkey under ContextMenuHandlers and sets the default value of that key to its GUID. So, for our simple extension, we'll create this key:

HKEY_CLASSES_ROOT\txtfile\ShellEx\ContextMenuHandlers\SimpleShlExt

and set the default value to our GUID: "{5E2121EE-0300-11D4-8D3B-444553540000}".

You don't have to write any code to do this, however. If you look at the list of files on the FileView tab, you'll see SimpleShlExt.rgs. This is a text file that is parsed by ATL, and tells ATL what registry entries to add when the server is registered, and which ones to delete when the server is unregistered. Here's how we specify the registry entries to add so Explorer knows about our extension:

HKCR
{
  NoRemove txtfile
  {
    NoRemove ShellEx
    {
      NoRemove ContextMenuHandlers
      {
        ForceRemove SimpleShlExt = s '{5E2121EE-0300-11D4-8D3B-444553540000}'
      }
    }
  }
}

Each line is a registry key name, with "HKCR" being an abbreviation for HKEY_CLASSES_ROOT. The NoRemove keyword means that the key should not be deleted when the server is unregistered. The last line has another keyword, ForceRemove, which means that if the key exists, it will be deleted before the new key is written. The rest of the line specifies a string (that's what the "s" means) that will be stored in the default value of the SimpleShlExt key.

I need to editorialize a bit here. The key we register our extension under is HKEY_CLASSES_ROOT\txtfile. However, the name "txtfile" isn't a permanent or pre-determined name. If you look in HKEY_CLASSES_ROOT\.txt, the default value of that key is where the name is stored. This has a couple of side effects:

  • We can't reliably use an RGS script since "txtfile" may not be the correct key name.
  • Some other text editor may be installed that associates itself with .TXT files. If it changes the default value of the HKEY_CLASSES_ROOT\.txt key, all existing shell extensions will stop working.

This sure seems like a design flaw to me. I think Microsoft thinks the same way, since recently-created extensions, like the QueryInfo extension, are registered under the key that's named after the file extension.

OK, end of editorial. There's one final registration detail. On NT, it's advisable to put our extension in a list of "approved" extensions. There is a system policy that can be set to prevent extensions from being loaded if they are not on the approved list. The list is stored in:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Shell Extensions\Approved

In this key, we create a string value whose name is our GUID. The contents of the string can be anything. The code to do this goes in our DllRegisterServer() and DllUnregisterServer() functions. I won't show the code here, since it's just simple registry access, but you can find the code in the article's sample project.

Debugging the shell extension

Eventually, you'll be writing an extension that isn't quite so simple, and you'll have to debug it. Open up your project settings, and on the Debug tab, enter the full path to Explorer in the "Executable for debug session" edit box, for example "C:\windows\explorer.exe". If you're using NT, and you've set the DesktopProcess registry entry described earlier, a new Explorer window will open when you press F5 to start debugging. As long as you do all your work in that window, you'll have no problem rebuilding the DLL later, since when you close that window, your extension will be unloaded.

On Windows 9x, you will have to shut down the shell before running the debugger. Click Start, and then Shut Down. Hold down Ctrl+Alt+Shift and click Cancel. That will shut down Explorer, and you'll see the Taskbar disappear. You can then go back to MSVC and press F5 to start debugging. To stop the debugging session, press Shift+F5 to shut down Explorer. When you're done debugging, you can run Explorer from a command prompt to restart the shell normally.

What Does It All Look Like?

Here's what the context menu looks like after we add our item:

 [SimpleShlExt menu item - 3K]

Our menu item is there!

Here's what Explorer's status bar looks like with our fly-by help displayed:

 [SimpleShlExt help text - 3K]

And here's what the message box looks like, showing the name of the file that was selected:

 [SimpleShlExt msg box - 9K]

Copyright and license

This article is copyrighted material, ©2003-2006 by Michael Dunn. I realize this isn't going to stop people from copying it all around the 'net, but I have to say it anyway. If you are interested in doing a translation of this article, please email me to let me know. I don't foresee denying anyone permission to do a translation, I would just like to be aware of the translation so I can post a link to it here.

The demo code that accompanies this article is released to the public domain. I release it this way so that the code can benefit everyone. (I don't make the article itself public domain because having the article available only on CodeProject helps both my own visibility and the CodeProject site.) If you use the demo code in your own application, an email letting me know would be appreciated (just to satisfy my curiosity about whether folks are benefitting from my code) but is not required. Attribution in your own source code is also appreciated but not required.

Revision History

March 27, 2000: Article first published.
March 14, 2006: Updated to cover changes in VC 7.1. Removed OLE Automation-related code from the project to simplify things a bit.

Series Navigation: » Part II

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Michael Dunn

Software Developer (Senior) VMware
United States United States
Michael lives in sunny Mountain View, California. He started programming with an Apple //e in 4th grade, graduated from UCLA with a math degree in 1994, and immediately landed a job as a QA engineer at Symantec, working on the Norton AntiVirus team. He pretty much taught himself Windows and MFC programming, and in 1999 he designed and coded a new interface for Norton AntiVirus 2000.
Mike has been a a developer at Napster and at his own lil' startup, Zabersoft, a development company he co-founded with offices in Los Angeles and Odense, Denmark. Mike is now a senior engineer at VMware.

He also enjoys his hobbies of playing pinball, bike riding, photography, and Domion on Friday nights (current favorite combo: Village + double Pirate Ship). He would get his own snooker table too if they weren't so darn big! He is also sad that he's forgotten the languages he's studied: French, Mandarin Chinese, and Japanese.

Mike was a VC MVP from 2005 to 2009.

You may also be interested in...