Troubleshooting non-paged pool memory leak event id 2019 using poolmon

Event id 2019 generated by source srv usually indicates that the server is running short on non-paged pool memory, the non-paged pool limitation on Windows 2003 32-bit is 256 MB which is used by kernel and device drivers

In case the NP pool is overloaded, the system becomes slow and unresponsive and some software components cease to work normally (for example, IIS starts refusing connections).

The NP memory pool shortage can be caused by memory leaks in third-party software, malware, or generally overstraining the system with resource-intensive operations.

I had encountered with one such similar issue during my day to day support for one the client. The server was repeatedly going into hung state and generated event id 2019 “The server was unable to allocate from the system nonpaged pool because the pool was empty”

———————————————————————————————————————–
Log Name: System
Source: srv
Date: 6/16/2014 5:21:06 AM
Event ID: 2019
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: Mytestserver.tech.com
Description:
The server was unable to allocate from the system nonpaged pool because the pool was empty.
———————————————————————————————————————–

After performing analysis on the server I had found that this issue was caused by a third party driver installed on the server, please read below in detail to understand usage of poolmon

Use Windows Task Manager to check NonPaged Pool value. If it is high (>200MB on a 32-bit system), it makes sense to analyze its utilization and fine-tune the server.
taskmgr

 

 

 

 

 

 

 

 

 

 

 

 

Use of poolmon.exe will show the number of allocations and outstanding bytes of allocation by type of pool and tag passed into calls. Various hotkeys cause Poolmon to sort by different columns to find the leaking allocation type, use either ‘b’ to sort by bytes or ‘d’ to sort by the difference between the number of allocations and frees.

Here’s Poolmon running on a system where BLFP had leaked 445342 allocations and BCM0 had leaked 40 allocations.

nonpaged
Once identifying the tag name in the left column the next step is to find the driver file that is using it, this can be achieved by performing search using findstr command with the tag name in the location “c:\windows\system32\drivers” where most of the drivers are located. To know more about using findstr with tag please visit ms kb http://support.microsoft.com/kb/298102

BLFP and BCM0 tags were related to Broadcom network adapter driver which was very old and outdated that caused all the problems on the server. Performing upgrade of Broadcom drivers to latest version fixed this issue.

There are some known pool tag names listed in the MS Technet site, please have a look at them as this list is very much helpful when we troubleshoot such issues. http://blogs.technet.com/b/yongrhee/archive/2009/06/24/pool-tag-list.aspx

Advertisements

About asifkhandevadi

Hello, I have been working on windows since 9 years and currently working as windows, VMware and MS clustering SME at IBM. Whenever I get free time I participate in Microsoft forums and write some blogs to enhance my technical and communication skills through knowledge sharing. Please contact me on FB or Linkedin if you need any assistance on troubleshooting, implementation and virtualizaton.
This entry was posted in Windows, Windows Troubleshooting and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s