The history of the Windows user profile in EUC environments (1994-2019)

I’ve been told I bang on about Windows profiles a lot. Let’s fulfill that prophecy!

Last week James Kindon, a fellow CTP and an affable guy who often chews the fat with me on Slack whilst he is clearly burning gallons of midnight oil, wrote a blog post about Windows profiles in 2019, which we had some chatter about prior to publication. During aforementioned conversation I promised to write up my “history of Windows profiles” presentation I did a couple of years ago. So rather than ruin the reference to it in his post by back-burning it for ages in my usual fashion, I thought I’d better get on and do it, for once.

Introduction

So what precisely is the “Windows user profile”? It is essentially a set of configuration items that represent the characteristics, the “personality” of the user. In my eyes, the profile is created where applications, data and configuration items intersect, a collection of Registry and filesystem artefacts that are bound to the individual user’s session and shape the environment within that. It holds all sorts of data – views, shortcuts, wallpapers, screen savers, colour schemes, supplementary files, dictionaries, signatures, auto-complete files, MRU lists, cookies, history, toolbars, autotext, connection settings; the list could go on and on.

Now, more modern software, especially SaaS and other cloud-based services, have taken to storing the “profile” at the “server end” of things these days, which eliminates a lot of problems that we have traditionally encountered around this device-based global user profile. For instance, the nickname file in Outlook (the file that auto-fills your email addresses based around people you’ve already emailed) used to be stored in a file with an .nk2 extension, but it now sits at the “back end” with the user mailbox. This is the model commonly used with mobile applications too, so as we move further on, expect the user profile to become less all-encompassing that it has been in the past. However, we still use desktop PCs where installed software often interacts quite heavily with other applications, so I don’t think that the user profile, on a whole, will ever fully go away. We may adapt as the years go on to a newer way of addressing the profile, but unless every vendor is willing to store our user settings and preferences wholesale on their systems, we will at least see some user profile items hanging around for the foreseeable future.

User profile issues are a lot more common than you think and you’d be surprised how much of a dramatic effect they can have on user experience. There has been a tendency to relegate profiles and profile issues to the “meh” end of priority, but in my opinion, as something that is intrinsically tied to the user’s perception of their systems, you should disregard them at your peril. In a Windows 10 migration I did, user profile-related issues accounted for a very large percentage of calls to the service desk in the three months after implementation. Logon issues, file type associations, and view settings made up the lion’s share of these problems.

It’s important to treat the user’s profile with a bit of respect. It can be an integral part of that intangible metric that we call “user experience”. When you lose a user’s profile, it can often have a dramatic and disruptive effect on their working practices. Take them seriously!

History and evolution

Now back in the days of yore (if you are as old as me), there was no such thing as a user profile. In the Windows 3.11 days, this generally wasn’t a multi-user capable operating system. If you were unfortunate enough to have to share a machine with a user who tinkered, you could come into work one morning and find yourself confronted with their changes – and maybe they’d have done something like this…

….and enabled Windows 3.11’s infamous “Hotdog Stand” colour scheme – guaranteed to make you feel nauseous.

Windows 95 and 98 continued this theme of having no true user profiles (although you could enable them to a certain, limited degree by using Control Panel), but for true multi-user Windows (as in multiple users could log on to the machine singularly – for multiple users as in multiple sessions, you’d have to buy a product called WinFrame from an upstart called Citrix) then you’d have to be using Windows NT.

Windows NT provided user with a true user profile for the first time. Originally they sat in c:\winnt\profiles\%USERNAME%. There were folders in here that held settings that applied to All Users as well (originally, just desktop and Start Menu items), and also an environment variable (%USERPROFILE%) that mapped to the user’s profile root when logged in. Interestingly, the files used for the user’s Registry hives have not changed since the Windows NT days at all. Some of the more pertinent parts of the first user profile are shown in the image below.

Naturally, because Carl Webster is older than God’s dog, he has corrected my remarks around the lack of multi-user capability prior to NT4. There is obviously this

https://en.wikipedia.org/wiki/PC-MOS/386

and this

https://en.wikipedia.org/wiki/Multiuser_DOS

to take into account. However from my own perspective, I didn’t see a true multi-user platform from Windows until the NT4 days, but clearly yes, there were implementations of it out there in enterprise use. Thanks for the history lesson Carl!

Creating a profile

How is a user’s profile created? Well, typically (and I know there are now many caveats around this, but I said typically), it is created from a special folder under the profiles storage area called Default User (which was later renamed, but the principle remains the same). This contains a fairly-vanilla set of configuration items which are used to build the profile of each user logging on to the machine that doesn’t already have a profile.

When a user logs on to a machine and they get a new profile created from the default, this is what happens

• Check for pre-assigned profile
• If none found, Registry hive is created under HKEY_USERS
• The profile folder is created for the user in a subfolder of the user profiles root folder with %USERNAME% as the folder name. If for some reason that folder already exists, then a folder will be created in the format username.computername (for non-domain endpoints) or username.domain (for domain endpoints). In the event there is also one of these folders, a number will be appended to the end of the folder name incrementally
• The contents of the default profile folder are copied into the new user folder
• Filesystem security is set on the new profile folder (depending on the settings in play, this may be exclusively for the named user and the SYSTEM account, or may also append the local Administrators group)
• The ntuser.dat hive from the default profile folder is merged into the new Registry hive
• Security is set on the Registry hive in line with that on the filesystem

This behaviour can be modified and/or overridden by a number of settings and/or products, but for a default Windows installation, this is what would be expected to happen.

Another point about default profiles is if you put a folder in the NETLOGON share of your DCs called Default User, a domain-joined device will look in this folder first for a default profile before using the one local to the machine. Obviously, this has the caveat of meaning that every user in the domain will try to use this default profile. If the endpoint is Windows 7/Server 2008 or higher, it will look for the appropriate suffix for this folder name dependent on the OS (so Windows 7 would look for \NETLOGON\Default User.v2, etc.). There are more details further down this post about profile versioning.

Customizing the default user profile is a great way of deploying base settings to your users and for speeding up logons – I’ve already written an article that covers this. I also find the Citrix User Profile Management “template profile” setting really useful, if you’re using UPM, because you can create a default profile in a file share and redirect certain groups of devices to different ones. However, you can also use FSLogix Redirection to create yourself multiple default user profiles on the same device, should you be so inclined – I should have a quick article detailing this available very soon.

Types of profiles

There are several different types of profile that can be used

A local profile is the commonest type of profile – a user profile that is stored locally on the device. They are for this reason the fastest to load and give the best performance, in general. However, the obvious problem for them is that they cannot be ported between user devices – each time the user logs on to a new device, a new, standalone copy of their profile will be created and saved.

Roaming profiles were the first attempt to solve this problem. The profile is stored on a file server SMB share and copied to the device at logon, and then back to the file share (with all attendant changes within) when the user logs off. This is typically defined in AD on the user, or if you were on a Windows NT domain, on the user object properties with User Manager.

A mandatory profile is a different type of roaming profile. It is still stored on a remote share, but it is read-only (renaming the ntuser.dat file to ntuser.man makes it read-only). The idea is that any changes the user makes in-session are discarded at logoff, making their session identical at each logon – ideal for kiosk or educational environments. A different type of mandatory profile called a super-mandatory profile was added in Windows 2000, which meant that the user would not be able to log on at all if the mandatory profile was available. You activated a super-mandatory profile by changing the name of the profile share to have a .man extension as well as the ntuser.dat. So if your profile was stored in a shared folder called \\SERVER\PROFILESHARE and had an ntuser.man, the profile would be standard mandatory. However if you stored it in a folder called \\SERVER\PROFILESHARE.MAN (the folder name, not the share name, for posterity) and it contained an ntuser.man, the profile would be super-mandatory. Setting up mandatory profiles requires a good bit of fine-tuning – see here for some helpful details.

A temporary profile is given to a user when, for whatever reason, there is a problem loading a user’s profile or a specified profile is unavailable. The temporary profile is created from the default user profile as usual, except it is discarded at logoff and a message is shown to the user advising them that their profile is temporary. To troubleshoot the reason for it, check the event logs on the device and act on the information within.

One other profile type that it is maybe worth mentioning is the “hybrid” profile, although this is not listed above as technically it is not a native Windows profile type. A hybrid profile is typically created by using a mandatory profile as the base for all users logging in to the device, and then layering profile settings over the top of it, typically from a piece of profile management software (Ivanti were possibly the first company to champion the terminology, although I may be mistaken). Hybrid profiles are often referenced in design documents, although as said before, they aren’t a genuine native Windows profile type.

Windows 2000-XP/2003

Windows 2000 (who remembers Windows 2000 fondly? I think it was a much better operating system then it was given credit for) started the evolution of the Windows profile with the following changes.

Let’s run through some of these points:-

• The base profile folder moved from c:\winnt\profiles to c:\Documents and Settings
• The %USERPROFILE% variable was updated to reflect this change
• New environment variables were introduced – %ALLUSERSPROFILE% for the c:\Documents and Settings\All Users folder, and %APPDATA% for %USERPROFILE%\Application Data
• Personal became My Documents
• The super-mandatory profile type functionality was introduced
• The Cookies folder arrived (because the Internet was now a thing!)
• A Local Settings folder was put into %USERPROFILE%, ostensibly for “non-roaming” data
• Processing of roaming profiles was improved – filesystem entries now checked timestamps before copying them back, instead of simply copying them back en masse as in the Windows NT days. This addressed some of the “last writer wins” issues (although not all, which we will discuss a little later)
• Profiles were now required to register themselves in the Registry at HKLM\Software\Microsoft\Windows NT\CurrentVersion\ProfileList (they still did this on NT4, but this entry wasn’t tied to the user profile)

The next iteration of the Windows OS was XP and Server 2003, which to be honest didn’t add very much from a profile perspective – the few changes that were are shown below.

There were some changes for the GPOs for roaming profile support at this time, which form the basis of the management policies we have for them today. In fact, probably the only pertinent one to be added since then is probably the “Allow deployment operations for special profile types”, which allows UWP apps to be provisioned into pre-existing roaming profiles on Windows 10.

Windows 7/Vista/2008/2008 R2

So now, as we moved on to Windows 7 and Vista and their server-end equivalents, we saw the most dramatic amount of changes in the profile.

There’s a lot to take in here, so let’s break it down:-

• C:\Documents and Settings was eschewed in favour of C:\Users, which has remained to this day (so far!)
• The Default User folder became just plain old Default
• %USERPROFILE%\Local Settings was merged in with Application Data, moving to %USERPROFILE%\AppData\Local and being mapped to %LOCALAPPDATA% as a variable
• %USERPROFILE%\Application Data was merged as above, moving to %USERPROFILE%\AppData\Roaming and being mapped to %APPDATA% as a variable
• %USERPROFILE%\AppData\LocalLow was added, intended to hold local non-roaming data that ran with a low integrity level (thanks to Pete Jones for confirming that this was introduced for Protected Mode in Internet Explorer)
• %ALLUSERSPROFILE% was remapped to %SYSTEMDRIVE%\ProgramData rather than c:\Documents and Settings\All Users
• Now, very importantly, many applications had been written which used the old Windows 2000/XP profile folder areas such as Default User, All Users and Application Data – which had just all been moved. To compensate for this, Microsoft filled the profile with junction points which could redirect applications accessing them to the new areas. In a nod to our reliance on legacy software, these junction points still exist in a standard Windows user profile today
• Because of all these changes, Windows 7+ user profiles were now no longer backwards compatible onto Windows NT/2000/XP machines, and the first profile version increment was born. A user logging on to a Windows 7 machine would have a profile folder marked with a .v2 extension “expected” by the operating system when logging in. So for instance, if a user had a roaming profile configured with a path of \\SERVER\SHARE\%USERNAME%, when logging on to a Windows XP machine, the system would look for a folder matching the exact path. However, when logging on to a Windows 7 machine, the system would append the .v2 to the path it searched for, so with \\SERVER\SHARE\%USERNAME% as the profile path on the user object, it would map to a filesystem path of \\SERVER\SHARE\%USERNAME%.v2

Another pertinent change at this level was that deleting a profile was no longer as simple as removing the user’s profile folder from the filesystem. Now, you had to remove the folder and the ProfileList entry from the Registry in HKLM\Software\Microsoft\Windows NT\CurrentVersion\ProfileList\[USERSID] (where USERSID matched the user’s SID). Often, you would see logon failures or temporary profiles caused by an orphaned entry in this key called [USERSID].bak – deleting this while logged on as an admin would resolve the issue. From hereon onwards, you had to delete profiles using the System applet in Control Panel, rather than just binning the user folder. Although there are good third-party tools that can delete them from the command line as well, such as delprof2 from Helge Klein.

Windows 8/8.1/Server 2012/Server 2012 R2

So moving on to the forgotten generation of modern Windows, we again saw a very small number of changes

• My Documents became just Documents
• The Start Screen heralded the rise of weird Windows filesystem entries that behaved in weird and annoying ways, using appsFolder* files in %LOCALAPPDATA% to hold the data required for this part of the UI
• From 8.1 onwards, the right-click Start Menu appeared (the WinX menu), and the entries for these were also stored in %LOCALAPPDATA%
• Originally these OSes were intended to be compatible with and use .v2 profiles, but Microsoft hastily issued a patch to correct a problem that was found, resulting in Windows 8/2012 profiles being marked as .v3, and 8.1/2012 R2 profiles as .v4. So not only incompatible with older OSes, but not even compatible with the updated version of itself.

Windows 10/Server 2016/Server 2019

And finally we arrive at the latest set of iterations of the Windows OS. There are a few changes worth noting in here as well:-

As said previously, the junction points for backwards compatibility still exist within the user profile

In particular on Windows 10 (and to a lesser extent on Server), Microsoft’s much-maligned UWP apps install themselves within the user profile. This causes a lot of problems from a profile management perspective, but apparently it is not a paradigm that Microsoft are heavily committed to for the future (hopefully true!)

For the early versions of Windows 10 (earlier than 1803) and Server 2016, the Start Tiles were configured using a database that sat in %LOCALAPPDATA%\TileDataLayer\Database (this was not the only part of the OS to adopt a database model, either). For 1803+ and Server 2019, Microsoft shifted focus and moved the data back to the filesystem and Registry in a folder called CloudStore.

The Windows 10 profile version was originally .v5, but the 1607 update incremented to a .v6. Currently, .v6 is used for the latest version of Windows 10 and both Server 2016 and 2019.

Profile versions

Here’s a summary of how the various profile versions sit according to OS. These types are not compatible with each other. Occasionally, you can take a downstream profile version, copy it to a new folder with a changed profile extension, and have a successful one-time upgrade, but you cannot go back.

Personally, I avoid mixing types at all as you can generate unforeseen issues. Some third-party management tools can make settings roam across platforms, though.

Roaming profiles

If you’ve read James Kindon’s article, he uses a pile of garbage as a visual image to represent roaming profiles. I do agree with the sentiment, as they are notorious for causing problems. Let’s dive into them a bit more here.

This is how a roaming profile works:-

The entire user profile is copied down from the network at logon and out again at logoff. Folder Redirection (another of my bugbears – see this article for more of a detailed rundown) is commonly used to mitigate problems caused by this approach, usually to do with large or bloated profiles. One of the common problems we see with roaming profiles though is last writer wins (see diagram below). This is particularly common in XenApp environments where users can sometimes have a hung or oprphaned session that is logged-off by an administrator after all other sessions have ended, and that writes the “wrong” changes back to the roaming profile server copy.

The changes introduced in Windows 2000 mitigated this to a certain extent because it checks the timestamps of the orphaned session before merging the changes back – but let’s not forget the ntuser.dat file. The Registry is essentially a filesystem within a file, and as such, it is always written to when logging off. Which is why last writer wins doesn’t have such dramatic problems these days like it used to under NT4, but you still get subtle issues and problems with persistence.

Without going in to all the possible issues (you can read much more of them in the Folder Redirection article), let’s just summarize some of the common problems with the roaming profile approach.

It is maybe worth mentioning that you can hack a roaming profile to work on Windows 10, by changing the ExcludeProfileDirs value in the Registry. More detail on that in this article.

Profile KPIs by OS

Let’s also take a look at how the performance of profiles for KPIs has changed down the years.

We can see that it’s been an upward curve in terms of performance right across the board – put modern apps like Chrome, Slack and Teams in the mix, and the pressure mounts even further.

Profiles here and now in 2019

So, what is the current thinking on profiles? Before I start my ramble around this subject, here’s some things we should be thinking about when we are talking profiles.

So what are my particular thoughts about profiles in the here and now of 2019? Firstly, I think roaming or mandatory profiles should be avoided because of their network locations and the way they behave. I wouldn’t include Citrix User Profile Management in this statement, despite the fact that it works in a similar fashion, because UPM gives you a huge amount of configuration that allows you to avoid many of the restrictions of roaming profiles, so UPM is still good for me when it is needed. If I do use a mandatory profile, for whatever reason (either for the “normal” reasons or to layer a third-party solution on top), I prefer to point the user object setting to a mandatory profile stored locally on the image and update that by copying it down from a central location. So network-stored roaming or mandatory profiles are a no from me – locally-stored mandatory profiles or UPM profiles, in the right cases they’re a yes.

However, I’ve written many times about how I would much prefer to use FSLogix Profile Containers (now owned by Microsoft) to address the problems of both persisting a profile and avoiding the bumps of Folder Redirection in one fell swoop. Certainly, given the fact that as well as Microsoft, Ivanti, Citrix and LiquidWare are all embracing the “virtual disk mount” as the silver bullet in this area, it appears to be the go-to way of addressing many of the profile issues. You get the performance of a local profile (almost), the central manageability of a roaming profile, the full portability from capturing everything out of the user profile root, and (if you go the Microsoft route), the ability to also handle multiple sessions and not pay any extra licensing costs.

For those who have invested heavily in solutions like Citrix User Profile Management, Ivanti User Workspace Management and LiquidWare ProfileUnity, this is not to say you should abandon them. A file-based solution can actually be faster to load when set up well. The key, though, is staying on top of this and keeping it trimmed. Environments where there are no specialist skills to deal with profiles and the overhead of maintaining them are ideally suited to container-based solution (assuming you can afford to pay for the storage required!) But if you absolutely need the fastest, most streamlined profile delivery solution around, or want to roam seamlessly between different operating system versions, you’ll need to look beyond what a container-based solution can offer (although I am working on some interesting stuff in this area – stay tuned!)

Citrix App Layering appears to be getting more serious about giving us the “user layer”, which is not just for profile persistence but also for user-installed apps – a subject I declared dead a few years ago, so some egg may be landing square on my face. I’m not sure that the user layer will be completely suitable for full-fat profile roaming, and I agree with James K that the overheads of standing up the App Layering infrastructure make this a solution for people who are already in bed with App Layering, rather than greenfield implementations looking for profile persistence. Jury is out, for me.

There’s no reason why you can’t host profiles in the cloud – I did a rudimentary article about using Azure Storage Accounts for FSLogix profiles, Jim Moyle did the same with Amazon S3. The endgame for Microsoft may well be to deliver FSLogix as a cloud service too, and I’ve seen many implementations of Citrix UPM and other third-party tools in cloud and hybrid cloud scenarious.

Resilience has sometimes been an issue for profile solutions, given that profiles are written to very heavily and there may be issues failing over in a timely fashion. I’ve had most success using DFS-R (particularly for UPM file stores), but it’s something that needs to be looked at carefully dependent on the solution chosen and the needs of the enterprise. Maybe one for a separate blog post 🙂

Summary

Managing user profiles is a fine art, and one which vacillates from pole to pole dependent on the needs of the business and the thinking of the time. There’s no “one size fits all” solution, even though I’ve preached the container-based method as possibly the one that will suffice for most. It’s a trade-off between speed, management, storage requirements and the needs of the user. One of the most hotly-debated areas is whether you should “compartmentalize” the profile to allow users to quickly reset parts of the profile rather than the entire entity in the event of an issue. This is arguably the area that gave AppSense (now Ivanti) so much traction in their early years when faced with standard roaming profile-based solutions. Currently I favour maintaining “hot” backups of user profile VHDs to allow an instant restore, but this again has overheads of management and storage. I’m working on some stuff around this subject as well, but don’t want to spill any details just yet – because then people would continually ask me when it will be finished 🙂

Think carefully about how you manage your user profiles. Like everything in IT, everyone follows the latest trend but seldom thinks about whether they should or not. Each approach to the user profile has its own merits and its own use cases. Use all the available knowledge from vendors and the community to come up with what works best for your business, your users and your applications. And have fun!