Citrix Xenapp/XenDesktop and Folder Redirection – the last word

Do we really still need to use Folder Redirection? Let’s do a deep-dive into it and find out.

Introduction

Folder Redirection has for so long been such an integral part of Windows enterprise infrastructure, it probably predates many of those working in the industry today. It is deployed almost without a second thought, but in the changing hybrid estates of 2018, is it still something we should be blithely adopting? Let’s do an in-depth look at the reasons behind, and technology used, for Folder Redirection, and see if there are ways we can do it more effectively today.

Excuse the header photo – I remember someone on Twitter once likening Folder Redirection to the Salem witch trials, because (and I quote), “they both seemed like a good idea to everyone at the time”. Indeed, Folder Redirection was once thought of as a necessary part of almost every EUC solution.

Background to Folder Redirection

So what was the genesis for Folder Redirection?

Quite simply, it was the roaming profile. Roaming profiles were an attempt to give a user in a Windows environment a consistent experience no matter where they logged on. Make a change to your user profile – new wallpaper, new desktop icon, etc. – and you could log on to a completely different device and see that change reflected. This idea resonated particularly with Citrix XenApp (probably called WinFrame or Metaframe at the time) admins, who used Microsoft’s RDSH technology (probably called Terminal Services at the time) to provide user sessions on hosted servers. No matter what server the user hit, their profile environment would be persistent. When XenDesktop and other VDI technologies came along later, the idea of using something like a roaming profile for persistence was applied there as well.

You can see from this diagram how a roaming profile works – the user logs on, their profile is copied from a network share to the local machine, and when they log out, it is copied back to the network share again. The roaming user profile is linked from the root of the user’s profile, but it only captures the content of the %USERPROFILE% folder and the APPDATA\ROAMING folder underneath. Anything in other folders (like APPDATA\LOCAL or APPDATA\LOCALLOW) is left behind, because the (original) idea behind these folders was that they contained data that wasn’t needed for roaming. The default exclusions for a roaming profile are configured in a Registry value called

HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\ExcludeProfileDirs

It is possible to customize your roaming profile to save more data including that from APPDATA\LOCAL by modifying this Registry value, but because it is an exclusions value, you need to specify every subfolder to exclude in order for this to function, which gives it a huge administrative overhead. Furthermore, some files in here are locked by the OS and cannot be copied, so even though I have written articles about using this key to save the entire profile before, it isn’t a great solution, nor a particularly scalable one.

Problems with roaming profiles

Roaming profiles traditionally have suffered from a number of problems, some of which, to be fair, are no longer so much of an issue as they were in the early 2000s. One of the most common issues is around last writer wins (shown in image below), particularly in XenApp environments where users may encounter “hung” or orphaned sessions.

Other common problems can be profiles becoming damaged and fail to load, especially where the OS or application maintains a hook into the files until deep into the logoff process. The configuration of certain policy settings can adversely affect performance. But as far as the inception of redirected folders were concerned, the primary problem of roaming profiles was the effect of large profiles on logon times and other KPIs.

This is a video (speeded up for posterity) of a Windows 10 desktop logging on with a roaming profile stored on a standard file server share. The profile size is 124MB.

The total time (as you can see) is 28 seconds (obviously if you’ve used all of the tips from my logon times article, this would be much faster :-)) Which is fairly reasonable for a profile of that size, all things considered.

Now let’s crank the profile size up by 1GB. I wrote a quick rudimentary script to generate files, which I have shared below should you wish to use it. Alternatively you can write something slicker 🙂

@echo off
setlocal
setlocal enableextensions

:: Set the “number_of_files” variable underneath to the amount you wish to create

set number_of_files=10

:: Set the “path_to_files” variable underneath to the path where you wish to create the files (no spaces)

set path_to_files=c:\filestore\homedrives\jrankin3\AppData\Roaming\
set total=0

call :LOOP

:END

set total=
set path_to_files=
set number_of_files=

endlocal
goto :eof

:LOOP

:: Uncomment the required fsutil command for the SIZE of the files you wish to create

:: Uncomment the line underneath for 1KB files to be created

:: fsutil file createnew %path_to_files%file%total%.tmp 1024

:: Uncomment the line underneath for 1MB files to be created

:: fsutil file createnew %path_to_files%file%total%.tmp 1048576

:: Uncomment the line underneath for 1GB files to be created

:: fsutil file createnew %path_to_files%file%total%.tmp 1073741824

set /a total=%total%+1

if %total% EQU %number_of_files% goto :END

call :LOOP

Once I’ve dropped a gigabyte of files into the profile, the logon time is affected as thus:-

So you can see, it’s sucky, the logon time is over a minute, it’s over twice as long. Ten times the profile size doubles the logon. This is the kind of problem that Folder Redirection came along to solve.

Now (and I know I’m getting a little tangential here, but bear with me), this is an interesting time to look at the effect you get from having lots of small files in your profile, rather than several big ones. Firstly, I tried to repeat the above process with a total of a million 1KB files – and my machine totally crashed. Literally, the logon never completed. Don’t try that at home.

It turns out that having more than around 10,000 files in a single NTFS folder is really bad, from this perspective. Upstream SMB versions allow lots of files to be handled more optimally, but even so, you’d be hard pressed to get high up the five-figure level as far as total files in a folder is concerned. With regards to 1KB files, I found that up to 3000 (that’s a grand total of 3MB of small files!) is about the best I could make do with.

So let’s not forget that we’re dealing with 3MB of files here as opposed to 1GB – here’s a table of logon times for 1GB in a single 1GB file, 1GB in ten 100MB chunks, and finally, a whopping 3MB in 3000 1KB chunks…

There you have it – a roaming profile with 3MB of small files takes over three minutes to log on, whereas a GB in smaller files takes around a minute. Now as I said, this terrible depreciation in performance might seem slightly tangential – but it’s very relevant when it comes to dealing with Folder Redirection, because if you’re relying on network shares, enumerating or copying files from these shares becomes a very big bottleneck pretty quickly.

So in this first instance, what we are primarily trying to solve is the uptick in logon times due to having a roaming profile that grows to a certain size. How does Folder Redirection help us with this?

Folder Redirection to the rescue

Folder Redirection works by directing specific folders from the user’s profile – usually ones that are large or contain lots of files – to network shares, rather than local folders. Therefore there is no need to copy them to or from the share – the profile now simply maps certain folders out to the network rather than processing them. A quick demonstration of this process is shown below.

By repeating the last test with the 3MB of files in 1KB chunks, but this time with them in a redirected folder, rather than being copied down to the local device, we can see the improvement in the logon processing.

So it appears, or at least it did, all that time ago, that Folder Redirection was something that solved a lot of problems for us, and that’s probably why it became so widely used.

How Folder Redirection works

So how do redirected folders actually work?

They’re actually configured by Registry values in the user hive, under HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders. These Registry values can be exposed directly, or through tech like System Policy Editor (long ago!), TweakUI, Group Policy Objects, or through a number of third-party tools like Ivanti, Citrix Workspace Environment Management, PolicyPak, LiquidWare ProfileUnity, VMware UEM, etc. The Registry values in this key refer to specific folders which can be configured for redirection.

Rather than editing the Registry directly, the most common way of handling this is through Group Policy Objects. On top of simply handling the pointer in the Registry, the GPO method (which is replicated into many of the third-party tools mentioned above) allows you to determine how to handle currently non-redirected data when a folder is redirected, and also specify what to do with redirected data when redirection is removed. It can also redirect to various different locations based around user variables and folder paths.

Is it better to edit the Registry directly or use the GPOs? If you edit the Registry directly, you are doing this in the simplest way – the SHGetKnownFolderPath API function looks up the Registry value and points the specified folder to it. Additionally, you can access redirection for twenty different folders (in the latest Windows version). With the GPO, there is more overhead of processing, as the redirection can be done around different parameters (mainly to do with the target), and you can only access thirteen folders for redirection. However when you redirect via GPO, as already mentioned above, you get many more options for control (see image below)

The list of available folders for redirection is shown below, along with a list of additional folders that will be redirected if you redirect the APPDATA folder (which maps to %USERPROFILE%\AppData\Roaming). Bear in mind that some third-party tools may support redirecting additional folders above and beyond those shown here (Ivanti UWM, for example, has 28 different options, although whether they are supported on all OSes is unknown), as well as possibly also providing the extended functionality of the GPO method.

Many people redirect directly through the Registry values, even though it’s not entirely clear if this is supported by Microsoft or not. To be fair, I’ve seen this used in large production environments and it appears there has never been a problem engaging Microsoft support. Certainly, redirecting by setting the Registry values directly is the most efficient way of doing this, because all it does is write to the Registry and read the paths from there.

You can also configure Folder Redirection via Citrix policies, which maps very closely to the same settings available in standard GPOs.

The main advantage of using this is to take advantage of Citrix-specific filters such as Delivery Groups, Tags, etc. that are not available within standard GPOs, allowing for more granularity around the application of the settings.

Many people are aware that doing Folder Redirection to a mapped drive is a bad idea (because you never know if the drive will map in time – and besides, mapped drives are an archaic concept that should be dead and buried anyway). Also, if you’re still setting up logging to fdeploy.log, forget about it, because since Windows 2008 Folder Redirection only logs to the event logs.

So with all this in mind, that having folders redirected to highly-available file shares makes KPIs better, makes management and backup easier, and avoids the needs for file copies at logon and logoff, you might be forgiven for thinking…

…and why not? After all, we’re selectively redirecting a lot of the folders from within it, aren’t we? It’s an interesting point, and one which I want you to hold on to until a bit later, but in the days of roaming profiles when we first had this thought, there were a few blockers. Mainly that to redirect the profile path (which is stored in the Registry at HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList\[USERSID] under the value ProfileImagePath) is very difficult – you need to read in the SID and set the path very early in the logon process. But even if you do manage it, there are a bunch of issues you get.

If done this way (i.e. hacked into the Registry manually), it seems that certain Microsoft programs running all the time on the client computer can not tolerate the sudden loss of their data folders if the server goes down or the network is disconnected. Some portions must still be copied back and forth before the desktop appears so that these folders are available if the network-redirected folders go down. The Windows shell becomes very unstable if the link is lost in this situation. Additionally, I observed these issues when testing on Windows 10 XenDesktop:-

Certain files from %LOCALAPPDATA% (upps.bin being one) cannot be copied at logoff
A profile synchronization error will be shown at each logon and logoff
Profile removal, if configured, will fail
If profile is redirected to a highly-available SMB share, a failover will more often than not damage the profile contents

So at that particular moment in time, we appeared stuck with Folder Redirection as an integral part of our deployments. Unless of course, there’s a third-party way of doing it, but as I said hold that thought till later.

Folder Redirection is often seen in conjunction with Offline Files, mainly for laptop users. Offline Files offer a way to cache folders (especially those ones that have been redirected) locally to the device for the situations when the redirected shares are not available. Anyone who has ever used Offline Files may well be familiar with the fact that it is an absolutely godawful piece of technology. Again, in the absence of anything better, it is quite usual to see Offline Files in use in the enterprise.

Folder Redirection problems

Unfortunately, Folder Redirection can, in some situations and configurations, bring a plethora of problems of its own.

The main ones relate to redirection of the APPDATA folder, which is a can of worms all in itself. This folder is where (generally) application data and configuration items are written to. Redirection of this folder can cause application instability, poor performance, network overheads – a whole gamut of different issues. It’s a very contentious issue, redirection of the APPDATA folder, and one which you can probably find a lot of Internet debate about without contributing further to it here. Here’s a quick example of a redirected APPDATA causing problems opening PDF files from the browser:-

It’s not just APPDATA, though – in the XenApp and XenDesktop world especially, I’ve seen a lot of problems where folders like Desktop have been redirected. Mainly, it causes performance issues as the Desktop is constantly refreshed across the network when interacted with. In this example, I’ve put 45 10MB files locally on the desktop of the user. Here, I will show the user interacting with them when they’re stored locally…

…and as you can see the interactions are fine, the user can pull up a context menu easily, there is no appreciable load on the system. Now let’s give the user a redirected Desktop folder and attempt to repeat the same tests…

…and you can see we now have huge CPU spikes, a hang in the Explorer process, and general difficulty for the user interacting with files on their desktop. Imagine this extrapolated out across a XenApp server with multiple users – the portents aren’t good. Redirecting one particular folder – the Desktop in this instance – has become a straight trade-off between good logon times, or good in-session performance.

The point I am trying to make is that Folder Redirection was brought along to solve some problems that we had, but over the years has simply brought us other ones. Redirecting any folders with lots of files in soon becomes an issue (Start Menu, Recent Items, Cookies, etc. – all of these I’ve had some sort of problem with in the past). AppData redirection is a whole problem in itself – only last week I saw a Citrix installer crashing repeatedly until we stopped AppData redirection. Shawn Bass, Aaron Parker and Helge Klein did a series of blog articles and presentations a few years ago regarding AppData redirection in particular, which should give you some further reading on it should you wish to do so.

We got around these issues by adopting different methods for retaining AppData and other high-volume folders. Ivanti UWM (particularly under its AppSense brand), Citrix UPM, RES, LiquidWare – there were many vendor solutions that tried to fill the hole and solve the problems of both roaming profiles and Folder Redirection. And to an extent many did, but there were always overheads of cost, infrastructure, maintenance and resource associated with them. And we still leaned on traditional Folder Redirection to some extent – we simply changed how we dealt with the more problematic examples of this.

We have used Folder Redirection as a crutch for a long time for the problems of roaming, and combined it with Offline Files when users are mobile. What we need is a different solution, or at the very least a change in the traditional approaches. I generally see enterprises redirecting everything, or redirecting everything except APPDATA because of performance concerns, and using a third-party tool (UPM, UWM, ProfileUnity, etc.) to manage the application data. But that just raises other problems – such as the Desktop one – which result in us pushing more back towards the storage of data in the profile, and even with tech such as streaming, just exacerbates the issues with logons and logoffs and other KPIs. Generally, we are vacillating between the problems of the roaming profile, or the problems of redirected folders, or the problems of managing a third-party solution. Is it time for a root-and-branch rethink of what we are trying to solve?

Requirements

So specifically, what is it we want to do? Taking the approach of Marcus Aurelius – what is my goal in itself?

Avoid the issues associated with large profiles or profiles that have many files
Avoid the performance impact of redirection on folders like Desktop, Start Menu and AppData
Save all the user settings that are required
Reduce any storage overhead
Support mobile users, preferably without a reliance on Offline Files

It’s also important to split our user access folders into two camps – high-volume and low-volume. High-volume are folders that have lots of small files, have a lot of read or write activity, and/or are generally refreshed or enumerated regularly by the shell or application. Think AppData, Desktop, Recent Items, Start Menu, Favourites and the like. Low-volume are folders that are accessed infrequently, usually data areas, so are typically read or written at the user request and only refreshed when the user accesses them through an application or folder. Examples of low-volume areas are Documents, Pictures, Videos, Music, etc. Delineating folders in this way allows you to deal with them differently.

The key to achieving your primary goals, at least, is to be able to put your target folder onto the network so it is centrally maintained and managed and reflects all the user changes for multiple devices – without the overhead of file copies and multiple reads and write, but also allowing the operating system to address it as a local folder.

As an example, let’s deal with the redirected Desktop folder we showed above, because that exhibited some really aberrant behaviour we can easily recreate.

Attacking the issue

Now, there is a current proclivity for the usage of mounted VHDs for dealing with various problems within EUC enviroments, which, unless you’ve been living under a rock for the last few years, you may well have noticed. VHD mounts bring a plethora of advantages, in that they are, essentially, a single file mounted across the network (despite being able to contain lots of files), which allow more efficient reads and writes. They also can then be centrally managed. So they may well lend themselves pretty handily towards a new kind of Folder Redirection, perchance?

At time of writing, most of the VHD mounting tools out there either mount the entire profile from the %USERPROFILE% root, or just specific large files, such as OST files. Probably the best-known is FSLogix (which coincidentally has just been acquired by Microsoft, so it’s clear that the approach is valid), although Ivanti, LiquidWare and Citrix also have functionality in this area. Now, there currently isn’t a vendor out there that allows you to redirect a single, specific folder to VHD which we want to do for this demonstration (although you could maybe mount it using PowerShell, but there are a lot of problems with elevation around this, so we’ve elected to simply hack our way around this instead) – update, Citrix now have the Profile Container setting in UPM which can do precisely this, although on a single-session basis only. FSLogix have a feature called Office365 Containers which simply redirects the Outlook OST file to a VHD on the network. But what you can do is specify a path to the OST file via Registry or GPO – and change this to %USERPROFILE%\Desktop, so now your previously-redirected folder is now mounted to a VHD file. Crafty!

Just in case you think I’m full of it, here’s the VHD created for the Desktop folder mounted so we can browse the filesystem and see the huge clump of temp files we generated earlier 🙂

So once we’ve done this, let’s repeat the test we did earlier (interacting with the Desktop during the session) and see if it is any better

What we can see is a small spike in CPU, but none of the massive ones we observed with the redirected folder, and certainly none of the hangs or delays we got in the shell process.

This solution extrapolates out to all of the problems we’ve had with redirected folders – AppData doesn’t generate crashes, installers don’t die, Start Menu search doesn’t hang on us, in fact mounting what we previously would have redirected to a VHD does the trick all round. Which is pretty cool….

…but the problem is that no vendor currently makes a feature that can mount individual folders to VHDs – I thought Ivanti were going to do this, but so far I haven’t seen this in their UWM product. You can do it with PowerShell, but it’s fiddly. But that’s probably not a reason to worry, and that’s why I told you earlier to hold that thought, because – why not just redirect the entire profile?

Now you can’t do this very well just by hacking the Registry, but you can do it with Microsoft User Profile Disks, FSLogix or LiquidWare Labs ProfileDisk – and given that Microsoft now own FSLogix, I’d expect UPD and FSLogix to merge together to produce a new product. So essentially, we redirect everything to a network location on a user level (you can even do it to the cloud, should you wish) which covers all of the high-volume redirections (user config items) I specified previously.

For the low–volume stuff (user data), then there is simply no substitute for an EFSS (Enterprise File Share and Sync) solution. OneDrive, ShareFile, DropBox, Box, whatever you want, use it here. This redirection normally points you at the folder in the local profile which is encapsulated in the profile VHD mount anyway, so literally the only Folder Redirections you are doing are not even on the network, as far as the OS cares. If you do need to redirect to a specific “home share” or folder, then you can do it here, but the benefits provided by having an EFSS client in play are really useful and should be taken advantage of.

So ideally, I like my approach to Folder Redirection to now look kind of like this.

The Downloads folder I use as a scratch area, which is never persisted. Users always need somewhere to dump things, so I always point it local and discard at logoff. Documents, Pictures, etc. is sent to EFSS or stored on a standard file share, and all the other folders are encapsulated in the VHD-mounted user profile which is stored on fast storage. Put bluntly – file-based Folder Redirection for config items is no more, it’s hopelessly inefficient, and does nothing but cause problems. Block-level VHD mounting Folder Redirection, for user config items, is the way forward, and ideally your EFSS data redirection should point into this too, because it gets around sync and storage issues.

There are just a couple of caveats to this approach when compared with our list of requirements. Firstly, storage. When you are redirecting the entire profile and consequently all the previously-redirected folders within it, you’re looking at some serious storage. Combine a Windows 10 profile with a sizeable OST and the entire contents of an EFSS share and we can easily be in the 5-10GB mark. You can take the attitude that storage is cheap (after all, it is these days, especially in the cloud) and just suck it up, or you can try to perform some exclusions to drop the size of the profile. The problem around exclusions (which hopefully I may be releasing a co-operative article on soon that can be referenced to help you out) is the overhead that comes with maintaining them. In my opinion, it’s best to adopt a bit of both – don’t be stingy with your storage allocation, but remove the obvious crap (like Chrome caches and the like, which the future planned article should help with identifying).

The other consideration is offline use. Now, FSLogix can handle offline situations by holding a copy of the mounted VHD locally for offline use. User Profile Disks and ProfileDisk, however, can’t (last I heard). Until last week, it would have been a case of forking out the extra for FSLogix to accommodate this – but given that Microsoft have now acquired them, I’d expect to see offline capabilities in UPD at some point, so you should be able to adopt this “new” method of dealing with redirected folders and factor in mobile users too.

Summary

We should forget Folder Redirection, as it was. It’s a legacy crutch we used to deal with legacy issues and it has caused us in the Citrix and RDSH/VDI worlds nothing but issues. Really, managing profiles (which Folder Redirection has long been an integral part of) on a standard filesystem level is dead. Microsoft’s acquisition of FSLogix gives further ammunition to those who say that the user persona should be managed on a VHD block level, and I believe that Folder Redirection should be consigned to the dustbin of history and replaced with a container-based solution in this way. That’s the only way we can manage things centrally, whilst still addressing the original issues we had, and avoid causing ourselves further problems.

It’s maybe worth mentioning Citrix App Layering briefly here too, as the User Layer feature is now out of labs (finally) and can be used to mount a VHD for the user persona in much the same way. It does have some limitations, but can also capture things like user-installed applications, so if you’re into App Layering in a big way then using the User Layer to box around Folder Redirection may be an option for you.

But really, in much the same way I think network drives should be sent to the dustbin of history, we need to stop thinking about Folder Redirection in any of our enterprise solutions, because its day has gone.