Reposted from Federal Times…

Agencies must deliver plans for modernizing their records management policies by late March, President Obama said in a Monday memorandum.

The memo urges officials to digitize records whenever possible but notes that greater use of electronic communications has "radically increased" both the amount and diversity of information that agencies must manage. Although that surge risks overwhelming agency systems, Obama said, technology "can make these records less burdensome to manage and easier to use and share."

In their plans, agencies must describe how they will improve or maintain their records management programs, particularly in regard to email, social media and other electronic communications. They also have to explain how they will use cloud-based services and storage solutions, as well as spell out any gaps or provisions in existing laws or regulations that get in the way of better management. The Office of Management and Budget, National Archives and Records Administration, and Justice Department will use those reports to come up with a government-wide records management framework that is more efficient, maintains accountability by documenting agency actions and promotes "appropriate" public access to records, Obama said.

The initiative won praise from leaders of two watchdog groups, OpenTheGovernment and Citizens for Responsibility and Ethics in Washington. In a joint statement, the two organizations noted that 95 percent of agencies reported last year that they were at risk of losing electronic records. Obama’s memo "puts in place a structure to begin addressing the problem" they said, but cautioned that lack of money will be a major hurdle.

This is a great alternative for folks who can’t be out of the office for a full week or want to see what SharePoint development is all about. The first stop will be February 15-16, 2012 at the Museum of Science in downtown Boston, MA.

Check it out here:
Critical Path Training – SharePoint Training – Great SharePoint Adventure 2010 Roadshow, Boston MA, February 2012.

The answer is yes.  It’s actually quite simple.  There is a new field called “Declared Record” that holds the date on which an item was declared as a record.  If that field is null, then the item was not declared.  The only trick is that you have to use the internal field name, which is “_vti_ItemDeclaredRecord”.

Here is a screenshot of a document library containing expense reports for an accounting department.  Notice that there are 2 records declared, the “Automobile Expenses” and “Expense Claim Form” documents.

Here is another shot of a simple query in CAML Query Builder that finds all of the declared record items in the library:

Finally, here is the result of executing the query:

So, there you have it.  Easy as pie.

Well, for one thing, it means that companies will no longer have to purchase multiple systems in order to meet the de-facto compliance standard.  This could be a huge coup for GimmalSoft, because it will now enable records managers to work directly with work-in-progress documents that are stored within SharePoint instead of having to manage records separately.

As a point of reference, I have seen the GimmalSoft tools, and I have to say they are impressive.  Definitely worth considering, just based on ease of use and the core functionality they provide (stay tuned for an upcoming series of posts).  From what I’ve seen so far, the tools take full advantage of the SharePoint user experience paradigm, which is saying a lot considering some of the other tools that are out there. 

With (DoD) 5015.02 certification under their belt, GimmalSoft is now poised to usher in a new era in the RM tools market, so this promises to be an exciting year in the records management space.  You can read the press release here.  For a more in-depth review of the GimmalSoft Compliance Suite, check out this piece by Bruce Miller at RIMtech.

In the meantime, I wanted to alert you to a bug in the SharePoint OM that I found along the way that might be stumping some of the more adventurous among you.

THE PROBLEM

In a nutshell – there is no way to remove a custom record declaration handler for a site collection via the prescribed methods available in the SharePoint object model. 

For instance, if you create a site-scoped feature that implements a custom record declaration handler and then registers it in your FeatureActivated event receiver by calling RegisterCustomCodeForRecordDeclaration(), you cannot then unregister your handler when the feature is deactivated by calling a corresponding Unregister method.  Why?  Because that method doesn’t exist.  The way I believe it was supposed to work was that you’d call the same method again from your FeatureDeactivating override, but passing empty strings for the assemblyName and className parameters. 

The problem is that this method has a bug that prevents it from properly unregistering your code.  The end result is that your custom declaration handler method still continues to be called even after your feature has been deactivated, potentially causing ALL future attempts to declare or undeclare records in the site collection to fail.

As an example, consider the following code:

using Microsoft.Office.RecordsManagement.RecordsRepository;
using Microsoft.SharePoint;

namespace SharePointArchitects.RecordsManagement
{
    public class MyDeclarationHandler : IRecordDeclarationHandler
    {
        public static void Register(SPSite site)
        {
            Records.RegisterCustomCodeForRecordDeclaration(site,
                typeof(MyDeclarationHandler).Assembly.FullName,
                typeof(MyDeclarationHandler).FullName);
        }

        // Called whenever a record is declared.
        public RecordOperationResult OnDeclare(SPListItem item)
        {
            RecordOperationResult result
                = RecordOperationResult.ContinueRecordProcessing;
            // ... custom code ...
            return result;
        }
    }
}

Here we have a typical implementation for a custom record declaration handler that exposes a static method to perform the registration of the class into a specified site collection by calling the RegisterCustomCodeForRecordDeclaration method.  As you can see, it passes the SPSite object along with the assembly and class name. 

Since there is no “UnregisterCustomCodeForRecordDeclaration” method provided by the SDK, you might assume (wrongly) that in order to remove the registration of your class, you would simply call the same registration method again with empty strings or null values for the assembly name and class name.  Seems reasonable, right?  Not so fast. 

Here is what happens:

• Passing null for the ‘site’ parameter throws an ArgumentNullException because custom record declaration handlers are always installed at the site collection level.   Therefore, it makes sense that the SPSite parameter would be required.
• Passing empty strings for either the assembly or class name causes ALL subsequent attempts to declare or undeclare records anywhere within the site collection to fail.

You heard me.

Users will now open the “compliance details” dialog and see the familiar “Declare as record” link.  They will click it and be prompted to confirm the declaration, and then nothing will happen.  They will do this repeatedly until they get tired, and then someone will call you.  They won’t be happy.

So what to do?  Digging a little deeper into the code reveals a minor bug in the RegisterCustomCodeForDeclaration() method.

THE BUG

An internal method is called during the record declaration process to retrieve the assembly and class name of any record declaration handler that may (or may not) be associated with the site collection.  It correctly checks whether the root web property bag contains an entry for the property "ecm_SiteCustomRecordsClass".  It then checks whether that value is null or empty.  If not, then it attempts to instantiate the declaration handler class specified.  If it’s empty, then it goes along its merry way declaring the document as a record.

 
However, there is a bug in the "RegisterCustomCodeForRecordDeclaration" method.  It ALWAYS assigns the "ecm_SiteCustomRecordsClass" property to a non-null value, even if the assemblyName and className parameters retrieved from the property bag are empty.  This is because it inserts a ‘|’ delimiter between the assemblyName and className property values.  Thus, if you pass empty strings for the assembly and class name parameters, the property is set to the value "|", resulting in an exception when the GetClassForCustomRecordsCode method tries to instantiate the class.  This in turn causes all subsequent attempts to declare or undeclare records in the site to fail.

THE WORK-AROUND

Until there is a fix provided by Microsoft, the only work-around is to manually remove the “ecm_SiteCustomRecordsClass” property from the root web of the site collection using the hard-coded property name as shown below. 

[ Note that this code may break if the property name ever changes. ]

public static void Unregister(SPSite site)
{
    // NOTE: There is no 'unregister' API call.

    // The only way to unregister the handler is to clear the
    // property "ecm_SiteCustomRecordsClass" in the root web
    // of the site.
    const string key = "ecm_SiteCustomRecordsClass";
    if (site.RootWeb.Properties.ContainsKey(key))
    {
        site.RootWeb.Properties[key] = "";
        site.RootWeb.Properties.Update();
    }
}

With these routines in place, you can easily register and unregister your custom record declaration handler from the Feature receiver, and all is well in SharePoint land.

using System;
using System.Runtime.InteropServices;
using Microsoft.SharePoint;

namespace SharePointArchitects.RecordsManagement
{
    [Guid("2a4480d1-a985-45ea-a297-47c5461d79c3")]
    public class RecordTrackerEventReceiver : SPFeatureReceiver
    {
        public override void FeatureActivated(SPFeatureReceiverProperties properties)
        {
            SPSite site = properties.Feature.Parent as SPSite;
            MyDeclarationHandler.Register(site);
        }

        public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
        {
            SPSite site = properties.Feature.Parent as SPSite;
            MyDeclarationHandler.Unregister(site);
        }
    }
}

I gave 2 talks, which I have uploaded to my SlideShare account and will post links to shortly.

Hats off to Steve Smith and all the great folks at Combined Knowledge. Guess I’ll have to start saving up for next year. What a blast!

I’ll also be speaking at the upcoming SharePoint Best Practices conference in London, April 11-13.  I had the distinct privilege of speaking at the London conference a couple of years back and it was truly a great conference all around.  This year, I’m giving two talks:

After London, I head straight over to Italy for the Microsoft SharePoint & Office Conference in Milan, April 19-21.  This will be my first time speaking in Milan, and I’m looking forward to making lots of new friends, although my Italian is admittedly pretty lame.

<shameless plug>If you really like this tool, then how about a little feedback?  Just click the “Rate/Review” link on the Visual Studio Gallery Page and give it the thumbs up.  Better yet – go ahead and write a quick review!  And if you have any thoughts about improving the schema, then drop me a line at john@johnholliday.net.</shameless plug>

For your coding pleasure, the following updates have now been incorporated into the CAML.NET Intellisense schema:

• Added Feature Identifier Lookups

There are many scenarios that require you to enter the GUID of an existing feature, but finding the correct identifier can waste a lot of time.  For example, setting up feature activation dependencies requires you to enter the correct feature id.   The CAML.NET schema now includes all of the built-in feature identifiers with a description of each one as shown in the following screenshot.

Note: The features are alphabetized by name (not by GUID), so you have to scroll through the list (just use the arrow keys) to find the one you want.

• Added Column Identifier Lookups

Similarly, when declaring a content type and you want to use one of the built-in columns,  you need to know the column GUID in order to reference it.  The schema now includes all of the built-in field identifiers as shown below.

• Fixed the SecurityBits Enumeration

The SecurityBits attribute should be a two digit string to determine read/write access.  The schema has been updated so that the enumeration includes each valid combination of digits (11, 12, 14, 21, 22, 24).

• Modified the UpgradeActions Element

Thanks to Chris O’Brien for providing updated documentation for the Feature Upgrade schema.  And be sure to check out his outstanding 5-part blog series on Feature upgrades.

For those of you already using the SharePoint 2007 version, you know the basic idea.  Extend the default CAML schema files with detailed annotations imported from the SharePoint SDK documentation so you get more information at your fingertips while editing CAML files.

But there were limitations on the amount of information you could get to show up in the Visual Studio 2008 IntelliSense tooltips.  I really wanted to include rich HTML with links and formatted scrolling text so you could basically have as much information as you need right there at your fingertips while coding.  So much better than interrupting your train of thought to go browsing through the online docs, right?

Enter Visual Studio 2010 and the simply wonderful Extension Manager.  I mean, this stuff is so cool it makes you want to quit fooling around with SharePoint and switch over to developing Visual Studio extensions full time.  (… NOT!)  But seriously, the VSIX framework really is kind of cool.  Combine that with WPF and you’ve got the makings of a really powerful tool for editing CAML files.

So, for the last 6 months or so, I’ve been working on an updated version that would take full advantage of the VSIX framework.  Here are the main features:

• Extends the default schema files with detailed annotations imported directly from the SharePoint SDK documentation.  This is similar to the 2007 version, but way cooler.
• Adds a custom WPF IntelliSense Presenter to the Visual Studio environment for a greatly enhanced developer experience.
• Automatically detects and downloads schema updates so you are always working with the most current information available.
• Links directly to the online SDK documentation for the currently visible element, so you can “drill down” into the documentation with a single click.  If no documentation link is available, it does a Bing search for the highlighted topic, which opens in the Visual Studio web browser.

You can download it directly from the Visual Studio Gallery on MSDN here.

Annotated Schemas for Richer Intellisense

The main idea behind CAML.NET Intellisense is to extend the core WSS schemas in two ways:

1. Add as much information as possible about each element and attribute using xs:annotation elements so that it pops up in context while editing; and
2. Identify and replace xs:string types in the schema with enumerated types so that the valid values for each attribute also popup in context while editing.

For example, using the out-of-the-box WSS schema that ships with SharePoint 2010, you get a drop-down like the following when editing a Feature element:

That’s better than nothing, of course, but it doesn’t say anything about what each Feature attribute is for, so unless you’re already familiar with it, you’d have to refer back to the SDK either online or in the CHM file. Kind of a drag, since you’ve got the list of valid attributes right in front of you. If you’re like most developers, this can be a real distraction because you tend to get lost in the details while searching for the information online when you should be busy coding your solution.

Instead, by using the annotated WSS schema provided by CAML.NET Intellisense, you can see the information you need right there in context. You only have to refer to the SDK documentation when you really need to learn more about the topic. Most of the time, you just need a hint, which is what Intellisense is all about, right?

New Enhanced IntelliSense Presenter

With all of the great new WPF-based enhancements that Microsoft has added to the Visual Studio 2010 user experience, why not take advantage of the improved extensibility layer and add a custom IntelliSense Presenter just for SharePoint? After all, some of the SDK descriptions of CAML elements are rather lengthy and get kind of hard to read in just a single line of text.

The CAML.NET Intellisense Presenter gives you a nifty way to get all that WPF goodness right where you need it most. Here’s an example of what you get while adding a FieldRef element to a ContentType:

Extending the Core Schemas with Enumerated Types

In addition to annotating the schemas with the corresponding SDK documentation for each element, we can go one step further and change the plain vanilla xs:string types to custom enumerated types so we get the list of valid choices for the attribute values. This comes in very handy and also helps to eliminate those nasty little typos that can be *really* hard to find (and the SharePoint CAML gets mighty snippy about).

There are lots of places in the SDK documentation where the expected attribute values are listed in the description, but are not enumerated in the schema. This again means that you’d have to go digging through the documentation to discover what SharePoint expects the attribute values to be. This can be an enormous waste of time – even if you have a pretty good idea of what the values are. Who can keep all that detail in their head?

So, CAML.NET Intellisense enumerates as many attribute values as possible so they are available at your fingertips when you need them. Each enumeration is also annotated, so you get contextual help for the available values.

Here’s another example that shows the enumeration of list template types when adding a ListInstance element to a Feature. This can be a huge time saver! How many times have you had to hunt down the table of available list template type identifiers?

Link Back to the SDK Whenever You Need To

Microsoft has invested heavily in new and improved online SDK documentation, so I thought why not link back directly to the SDK so you can drill down whenever you need to? Sometimes you just need a little hint, but at other times, you need more detail. CAML.NET Intellisense gives you the best of both worlds. Simply click the magnifying glass icon. If there is an SDK topic for the current item, it is displayed in the Web Browser tool window. If not, then it executes a web search for more information about the topic being displayed.

Stay Up To Date on the Latest Best Practices

As we all know, things change fast in our industry.  There are always new ideas, new approaches to stay on top of, new techniques for dealing with the many issues that can arise unexpectedly.

I pondered quite a bit on this one, but I felt it was very important to have a way for the extension to update itself automatically whenever new “best practices” become widely accepted.  Also, the MSDN documentation is extensible with community content.  Some of that content helps to clarify and add critical detail for various CAML tags and attributes.  I wanted a reliable way to make that information show up automatically in context, so that SharePoint developers don’t have to go looking for it.

With the extension installed, the customized SharePoint schema files are updated automatically (if a new version is available) whenever you startup the Visual Studio environment.   That way the schema authors (myself and a few dedicated MVPs) can regularly scan the SharePoint SDK and online SharePoint community resources and push new information out through the schema file.  This makes it easier for you to find the information you need, and puts it literally right at your fingertips.

Available Now from the Visual Studio Gallery

You can download the new version directly from the Visual Studio Gallery on MSDN.

Best of all, it’s free.  Free as the desert sand!  So what are you waiting for?  Get ready to ride!  Ride that CAML!  Yeah!

Sorry.

I’m just so excited it’s finally out there.

As always, your comments are welcome.

Faster.  Better.  Like a dromedary!

Anyway, I’m sitting there at the breakfast table thinking about content types, taxonomies, information policies and “ECM for the Masses”, and all of a sudden a poem starts to materialize in my brain.  Has that ever happened to you?  Just all of a sudden, a poem pops into your head?  Here goes:

Oh, what a tangled web we weave,
when content types we do conceive.
Derived from documents and pages,
Information policies, and retention stages.
Columns with mysterious fields,
to SharePoint must all workers yield!
To SharePoint, all our cares and woes,
Indeed, to SharePoint all our content goes.

You know, it really sounded a lot better on an empty stomach…