Insufficient System Resources Console Error Message

Mindwatering Incorporated

Author: Tripp W Black

Created: 04/02/2012 at 03:49 PM

 

Category:
Domino Server Issues Troubleshooting
Server Console

Issue:
Domino has error message when doing a system task (e.g. compact or fixup) on the console while processing one or more applications (e.g. mail files).
Insufficient system resources exist to complete the requested service

Error is typically seen on MS XP and MS Windows 2003 servers which are 32bit. I've never seen this on Linux even with a 32bit Domino as the 64bit Linux OS seems to be able handle the memory pool calls.


Possible Solutions.

Check that external 3rd party tasks or Domino-based tasks are not running on Domino files. This can include a system-based ncompact -b command run within a bat file outside of Domino instead of as a Program document within Domino. This is also a problem with A/V or "cold" backup programs that don't have exclusions for the Domino folders.

Applications basically hold the Domino files open and then trigger the error when a user opens an application via the IBM Lotus Notes client. The Db Cache allows faster repeat access to applications often used by keeping them in memory. Each open application uses up 2k for every 1mb of memory (or 4k before W2K SP2) in the Windows Page Pool.

This message is normally seen on the Domino console and not the Windows Event log. It can often lead to a crash as Domino keeps disk page files open for re-use. System applications (dbs) such as names.nsf, mail.box(es), log.nsf, etc, are always open, plus the recently used applications are cached. IBM has notes.ini settings to disable or reduce the file use cache as a "temporary workaround". I personally would not recommend killing the cache, but you can reduce it using NSF_DBCACHEMAXENTRIES=x to find the sweet spot.

Temporary Domino Work-around Options:
Disable the DB Cache in Domino using:
NSF_DBCACHE_DISABLE = 1
(I personally don't like this one, limit instead below.)

Limit the DB Cache in Domino using:
NSF_DBCACHE_MAXENTRIES = 10
(where 10 is the maximum apps to cache at a time, oldest from last use will fall off first. Make sure you keep the always opened ones, names.nsf, log.nsf, mail.box, etc.)

Create Program document that executes the following console command during back-ups or other intensive times:
dbcache flush


Change the Windows Page Pool in the registry:
1. Start regedit via run or a command prompt:
regedit or regedt32.exe

2. Locate the following exiting key group:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Memory Management

3. On the Edit menu, click Add Value

4. Enter the following new registry string value:
Value name: PoolUsageMaximum
Data type: REG_DWORD
Base: Decimal
Value data: nn

Example nn: 60
This number tells the system Memory Manager to trim at nn percent of PagePoolMax. The default is 80%. Entering 60 would mean only use 60% of the maximum pool size.

The memory management of MS Windows is apparently great enough to handle 80% utilization. Going over 80% seems to give this error. So what you have to do is protect Windows for getting to 80. With the /3GB switch off, MS recommends a maximum usage of the memory pool to 60% or 40% if using the /3GB switch.
(My Computer --> Properties --> System Properties box --> Advanced tab --> Startup & Recovery section --> Settings --> S & R dialog --> Edit --> [Operating Systems] section --> add/remove /3GB after the /fastdetect hardware switch)

I don't recommend the accompanying PagedPoolSize as setting to maximum (0xFFFFFFFF) means the system uses up as much memory as possible at the expense of the other services, which can easily lead to blue screens because of other starved OS resources.

____________

Other Software:

Issue on XP SP2:
Typically, the PC has the maximum amount of RAM that Windows 32bit can support or more (e.g. 4GB). The errors tend to take place during events such as A/V checks, hibernation, and other software disk scanners that tend to use a lot of contiguous memory proportional to the amount of memory the computer has.

For hibernation apply all fixpacks and updates to XP. See KB909095.
For issues with Trend or Symantec, search their sites for "insufficient resources". Typically, you will posts regarding poolsize updates, testing in "clean mode", etc. See the server section below.


Issue on Windows 32bit:
Issue commonly phrased as "Cannot do __________, Insuffficient system resources to complete the requested service".

In most cases the issue supposedly has to do with paged kernel memory. The issue arises when 80% is exceeded for the paged pool memory to the set maximum size. The causes are:
- More files are open than the memory cache manager can handle - the cache manager has exhausted the available paged pool memory or at least past 80% of it.
- The file that is trying to be read into memory (e.g. for backup, compact, etc) is larger than the service can access based on available memory. Memory is addressed in Windows 2003 in 4GB address space divided for 2GB OS and 2GB applications, assuming the /3GB switch has not been added to the startup (does 1GB for OS, and 3GB for applications).

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management
Create/update the DWORD tags: PagePoolSize & PoolUsageMaximum.

Notes:
PoolUsageMaximum should not be set higher than 60, unless the /3GB switch is enabled, in which case, the limit is 40.
The idea is to REDUCE this value to force the calling program to use less memory in the pool at once.
PagePoolSize is used to set the maximum pool size compared to other system resources especially something "unrelated" called a PTE.
ex: 0xFFFFFFFF (-1) to set the maximum resources.
BE VERY CAREFUL, especially with /3GB switch already enabled, as blue screens of death likely.


NTBackup.exe / Symantec Backup Exec:
Issue numbers typically 1450, 1130, 1220.
See MS KB 304101 and Symantec article TECH69002

You REDUCE This value as needed until the message goes away.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management
Create/update the DWORD tag: PoolUsageMaximum. Set to 60 and test. Next reduce to say 40 and test.

See:
http://publib.boulder.ibm.com/infocenter/domhelp/v8r0/index.jsp?topic=/com.ibm.help.domino.admin.doc/DOC/H_MANAGING_THE_DATABASE_CACHE_STEPS.html

See MS article 312362
For Backup Exec and Domino, see Symantec article TECH74275




previous page