Configure the file system cache in Windows NT

1 January 2000 by Snakefoot | Comment » | Trackback Off
Windows has a file caching mechanism which is tightly integrated with the memory manager. It helps disk performance by keeping the most recent files in memory, whether is is EXE or DLL files loaded by the memory manager or data files loaded by the running applications.

The filesystem cache can work in two different modes:
  • The default mode is that the filesystem cache only grows to a certain limit (8 MByte).
  • The other mode is that the file system cache can grow until it has taken all memory (Up to 1 GByte depending on available memory).
One should not change from the default mode on standard workstations as a single application can trigger the file system cache to take all available RAM (Ex. by copying a large file), and the memory of other applications will be paged to disk (Even more I/O)

To allow the file system cache to make use of "all" available memory:
  • WinNT4: Open Control Panel -> Network -> Change properties for "Server" service to use "Maximize Throughput for File Sharing" (Will also change Server service memory usage).
  • Win2k: Open Control Panel -> Network and Dial-up Connections -> Right-click Local Area Connection and select Properties -> Press Properties for service "File and Print Sharing for Microsoft Networks" to set "Maximize Data Throughput for File Sharing" (Will also change Server service memory usage)..
  • WinXP/Win2k3: Open Control Panel -> System-Applet -> Advanced-Tab -> Performance-Settings-Button -> Advanced-Tab and select "System Cache"-Option.
  • Registry:

    [HKEY_LOCAL_MACHINE \System \CurrentControlSet \Control \Session Manager \Memory Management]
    LargeSystemCache=1 (Default Srv. = 1, Default Prof. = 0)

Note if wanting better control than just size Large and Small then try one of these utilities:
  • Microsoft Windows Dynamic Cache Service - Microsoft have recognized that 64bit versions of Windows has problems controlling the size if the working set for the file system cache (Up to 1 TB). This service can control the size of the file system cache according to workload, which can be useful if not using Windows 7 that tries to fix this issue. More Info NTDebugging
  • SetCache - Sets a permanent upper limit for the file cache using the SetSystemFileCacheSize API (Win2k3/Win2k8/Vista only). Use a trigger to activate this tool to set file cache limits at startup.
  • SysInternals CacheSet - It can only reset the cache to a certain size from where it can grow or shrink again.
  • Uwe Sieber - NtCacheSet - Periodically resets the cache working set, just like CacheSet but does it with a specified interval.
  • Uwe Sieber - SetSystemFileCacheSize - Sets a permanent upper limit for the file cache using the SetSystemFileCacheSize API (Win2k3/Win2k8/Vista only). Use a trigger to activate this tool to set file cache limits at startup.
Note Win2k3 SP2 includes a new registry setting that controls how much physical memory, the file cache can use for data that has to be written to disk. By default the file cache can occupy half of the physical memory, so when copying a large file (>500 MByte) then it will read the file into the cache until it reaches the limit. When it reaches the limit it throttles the file reading until it has written the contents of the cache to disk. This can cause the memory manager to hang, along with all applications, so one should consider to limit the size of the write cache:

[HKEY_LOCAL_MACHINE \System \CurrentControlSet \Control \SessionManager \MemoryManagement]
SystemCacheDirtyPageThreshold = 100 (MByte; Default = 0; 0 = Half of physical memory)

More Info MS KB 920739 (Doesn't limit the size of the read cache)

Note the LargeSystemCache is allocated from kernel memory area, which is shared with the pagedpoolsize and systempages. So when limiting these from using max size, then it will allow the filesystem cache to reach its max size of 960 MByte (WinNT4 512 MByte), else it will be limited to 512 MByte.

Note not all applications will benefit from a large system cache. Many disk intensive applications (Like database systems) includes their own cache manager, and doesn't make use of the operating system controlled file cache. For MS SQL and Exhange the optimal value is to disable the large system cache.

Note enabling the large system cache can give stability issues, because it will configure the kernel area to allocate max memory for the file cache, while sacrificing the size of memory pools and number of page table entries. If using Unified Memory Architecture (UMA)-based video hardware or an Accelerated Graphics Port (AGP), then it will require page table entries to address the video memory. ATI recommends one doesn't enable large system cache to avoid data corruption. More Info MS KB Q895932

More Info MS Technet - File Cache Performance and Tuning (Good read)
More Info MS Technet - Optimizing your memory configuration
More Info MS KB Q232271
More Info MS KB Q228766
More Info MS KB Q837331

Credits ArsTechnica.com

Updated: 19 March 2009

Comments:

Comment by Glen Heitman - 15 May 2004 @ 1:02 Reply

Data coruption may occur if the large system cache feature is enable in Windows XP for ATI Radeon graphic cards. See www.ati.com/support/infobase/4217.html

Comment by Larry Miller - 17 March 2009 @ 17:12 Reply

Microsoft recommends that the LargeSystemCache setting be used only for systems that are primarily used as servers. For workstation use it will almost always be bad. The setting allows the system cache to grow to very high levels, even at the expense of memory for application use. For a file server this is desireable but can easily kill application performance. Use only when appropriate and after careful consideration of the implications and risks involved. When used on systems with ATI video cards, and other devices as well, it can be deadly. This is not a universal performance tweak as many “tweaking guides” have implied.

Larry Miller
Microsoft MCSA

Comment by Snakefoot - 19 March 2009 @ 10:43 Reply

Larry Miller wrote:
Microsoft recommends that the LargeSystemCache setting be used only for systems that are primarily used as servers. For workstation use it will almost always be bad

You absolutely right, and I have now updated the article to make it even more clear that one should not enable large file system cache on workstations.

Thank you for your contributions to this website

Comment by Robert Miles - 6 August 2010 @ 14:37 Reply

How does the addition of the SuperFetch feature when Windows Vista was released affect this?

I’ve found SOME information about the SuperFetch feature, but so far, not enough to try adjusting the registry variables that control it.

http://www.codinghorror.com/blog/2006/09/why-does-vista-use-all-my-memory.html

http://blogs.technet.com/b/markrussinovich/archive/2008/04/07/3031251.aspx

http://blogs.technet.com/b/askperf/archive/2007/03/29/windows-vista-superfetch-readyboost.aspx?PageIndex=2

Leave a comment


NB! Use the Forum for computer help and off-topic questions.

You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>