Investigating NSPR Logging

There are a lot of tests written in C++ and there are a lot of different ways that they log their results. Among the strings that one finds are ": PASS", "INFO PASS", "tests passed.", "... OK", and so on. And that is only for the successful runs of the tests.

I am investigating the use of the PR_LOG call defined in http://lxr.mozilla.org/mozilla/source/nsprpub/pr/include/prlog.h.

It turns out that you can turn on logging in modules that take advantage of the NSPR function but setting an environment variable. So, if I want to see the log output from a test, I can do:

$ cd dist/bin
$ NSPR_LOG_MODULES=all:5 ./TestCookie 2>&1 | grep -v WARNING

At this point, there is a lot of output. I can also do:

$ cd dist/bin
$ NSPR_LOG_MODULES=all:0 ./TestCookie 2>&1 | grep -v WARNING

These numbers are from prlog.h. I found this in a post about troubleshooting mail. Or rather, Tomcat found it.

PR_LOG_NONE = 0, /* nothing */
PR_LOG_ALWAYS = 1, /* always printed */
PR_LOG_ERROR = 2, /* error messages */
PR_LOG_WARNING = 3, /* warning messages */
PR_LOG_DEBUG = 4, /* debug messages */

So, 5 means "I want everything" and 0 means "please, be quiet". But there is still a lot of output when I use "all:0" above.

It turns out that this test is using printf. I do not even want to do a search for the number of cpp files we have using printf. I am sure there are a gazillion of them.

When you register with the logger, you have to give your module a name. This is an interesting list.

"Autodial" - netwerk/base/src/nsAutodialWin.cpp
"BMPDecoder" - modules/libpr0n/decoders/bmp/nsBMPDecoder.cppPRLogModuleInfo
"BayesianFilter" - mailnews/extensions/bayesian-spam-filter/src/nsBayesianFilter.cpp
"DOMLeak" - dom/src/base/nsGlobalWindow.cpp
"DeviceContextSpecGTK" - widget/src/gtk2/nsDeviceContextSpecG.cpp
"DeviceContextSpecQt" - gfx/src/qt/nsDeviceContextSpecQt.cpp
"DeviceContextSpecXlib" - gfx/src/xlib/nsDeviceContextSpecXlib.cpp
"DeviceContextXlib" - gfx/src/xlib/nsDeviceContextXlib.cpp
"DocLoader" - uriloader/base/nsDocLoader.cpp
"DocumentLeak" - content/base/src/nsDocument.cpp
"DrawingSurfaceXlib" - gfx/src/xlib/nsDrawingSurfaceXlib.cpp
"FontMetricsGTK" - gfx/src/gtk/nsFontMetricsGTK.cpp
"FontMetricsPS" - gfx/src/ps/nsFontMetricsPS.cpp
"FontMetricsXlib" - gfx/src/xlib/nsFontMetricsXlib.cpp
"GC" - nsprpub/lib/msgc/src/prmsgc.c:
"HelperAppService" - uriloader/exthandler/nsExternalHelperAppService.cpp
"IMAP" - mailnews/imap/src/nsImapProtocol.cpp
"IMAP" - mailnews/imap/src/nsImapProtocol.cpp
"IMAPOFFLINE" - mailnews/db/msgdb/src/nsMailDatabase.cpp
"IMAPOFFLINE" - mailnews/db/msgdb/src/nsMailDatabase.cpp
"IMAPOFFLINE" - mailnews/db/msgdb/src/nsMsgOfflineImapOperation.cpp
"IMPORT" - mailnews/import/comm4x/src/nsComm4xMailImport.cpp
"IMPORT" - mailnews/import/eudora/src/nsEudoraImport.cpp
"IMPORT" - mailnews/import/oexpress/nsOEImport.cpp
"IMPORT" - mailnews/import/outlook/src/nsOutlookImport.cpp
"IMPORT" - mailnews/import/src/nsImportMail.cpp
"IMPORT" - mailnews/import/src/nsImportService.cpp
"IMPORT" - mailnews/import/text/src/nsTextImport.cpp
"ImageXlib" - gfx/src/xlib/nsImageXlib.cpp
"InMemoryDataSource" - rdf/base/src/nsInMemoryDataSource.cpp
"JPEGDecoder" - modules/libpr0n/decoders/jpeg/nsJPEGDecoder.cppPRLogModuleInfo
"JSComponentLoader" - js/src/xpconnect/loader/mozJSComponentLoader.cpp
"JSDiagnostics" - dom/src/base/nsJSEnvironment.cpp
"LoadGroup" - netwerk/base/src/nsLoadGroup.cpp
"MAILBOX" - mailnews/local/src/nsMailboxProtocol.cpp
"MAPI" - mailnews/mapi/mapihook/src/msgMapiImp.cpp
"MCD" - extensions/pref/autoconfig/src/nsReadConfig.cpp
"MIME" - mailnews/mime/emitters/src/nsMimeBaseEmitter.cpp
"Mai" - accessible/src/atk/nsMai.h:
"Movemail" - mailnews/local/src/nsMovemailService.cpp
"MsgBiff" - mailnews/base/src/nsMsgBiffManager.cpp
"MsgPurge" - mailnews/base/src/nsMsgPurgeService.cpp
"NNTP" - mailnews/news/src/nsNNTPProtocol.cpp
"NNTP" - mailnews/news/src/nsNNTPProtocol.cpp
"NNTP" - mailnews/news/src/nsNNTPProtocol.cpp
"NNTP" - mailnews/news/src/nsNNTPProtocol.cpp
"NSJAVA" - modules/oji/src/nsJVMManager.cpp
"NTLM" - security/manager/ssl/src/nsNTLMAuthModule.cppPRLogModuleInfo
"ObserverService" - xpcom/ds/nsObserverService.cpp
"PALMSYNC" - mailnews/extensions/palmsync/src/PalmSyncImp.cpp
"PNGDecoder" - modules/libpr0n/decoders/png/nsPNGDecoder.cppPRLogModuleInfo
"POP3" - mailnews/local/src/nsPop3Protocol.cpp
"PangoFont" - gfx/src/gtk/nsFontMetricsPango.cpp
"PhGfxLog" - gfx/src/photon/nsRenderingContextPh.cppPRLogModuleInfo
"PresShell" - layout/base/nsPresShell.cpp
"QtGfx" - gfx/src/qt/nsGfxFactoryQt.cppPRLogModuleInfo
"RDF" - rdf/base/src/nsCompositeDataSource.cpp
"RenderingContextXlib" - gfx/src/xlib/nsRenderingContextXlib.cpp
"RenderingContextXp" - gfx/src/xprint/nsRenderingContextXp.cpp
"SMTP" - mailnews/compose/src/nsSmtpProtocol.cpp
"SOCKS" - netwerk/socket/base/nsSOCKSIOLayer.cpp
"StaticComponentLoader" - xpcom/components/nsStaticComponentLoader.cpp
"Syspref" - extensions/pref/system-pref/src/gconf/nsSystemPrefService.cpp
"Syspref" - extensions/pref/system-pref/src/nsSystemPref.cpp
"Test" - netwerk/test/TestBlockingSocket.cpp
"Test" - netwerk/test/TestCookie.cpp
"Test" - netwerk/test/TestIOThreads.cpp
"Test" - netwerk/test/TestProtocols.cpp
"Test" - netwerk/test/TestServ.cpp
"Test" - netwerk/test/TestSocketTransport.cpp
"Test" - netwerk/test/TestStreamChannel.cpp
"Test" - netwerk/test/TestStreamLoader.cpp
"Test" - netwerk/test/TestStreamPump.cpp
"Test" - netwerk/test/TestStreamTransport.cpp
"Test" - netwerk/test/TestUpload.cpp
"Test" - nsprpub/pr/tests/anonfm.c:
"Test" - nsprpub/pr/tests/instrumt.c:
"Test" - nsprpub/pr/tests/mbcs.c:
"Test" - nsprpub/pr/tests/nameshm1.c:
"Test" - nsprpub/pr/tests/ntioto.c:
"Test" - nsprpub/pr/tests/op_excl.c:
"Test" - nsprpub/pr/tests/randseed.c:
"Test" - xpcom/proxy/tests/proxytests.cpp
"TestCase" - security/nss/cmd/selfserv/selfserv.c:
"TestSocketIO" - netwerk/test/TestOverlappedIO.cpp
"TestSocketIO" - netwerk/test/TestSocketIO.cpp
"URILoader" - uriloader/base/nsURILoader.cpp
"UrlClassifierDbService" - toolkit/components/url-classifier/src/nsUrlClassifierDBService.cpp
"UrlClassifierStreamUpdater" - toolkit/components/url-classifier/src/nsUrlClassifierStreamUpdater.cpp
"Widget" - widget/src/gtk2/nsAppShell.cpp
"WidgetDraw" - widget/src/gtk2/nsAppShell.cpp
"WidgetFocus" - widget/src/gtk2/nsAppShell.cpp
"WidgetIM" - widget/src/gtk2/nsAppShell.cpp
"XRemoteClient" - widget/src/xremoteclient/XRemoteClient.cpp
"XftFontLoad" - gfx/src/gtk/nsFontMetricsXft.cpp
"XlibScrolling" - widget/src/xlib/nsWidget.cppPRLogModuleInfo
"XlibWidgets" - widget/src/xlib/nsWidget.cppPRLogModuleInfo
"cache" - netwerk/cache/src/nsCache.cpp
"canvasGLES11" - extensions/canvas3d/src/nsCanvasRenderingContextGLES11.cpp
"clock" - nsprpub/pr/src/misc/prinit.c:
"cltsrv_log" - nsprpub/pr/tests/cltsrv.c:
"cltsrv_log" - nsprpub/pr/tests/provider.c:
"cmon" - nsprpub/pr/src/misc/prinit.c:
"cookie" - netwerk/cookie/src/nsCookieService.cpp
"counters" - nsprpub/pr/src/misc/prcountr.c:
"cvar" - nsprpub/pr/src/misc/prinit.c:
"expatdriver" - parser/htmlparser/src/nsExpatDriver.cpp
"frame" - layout/generic/nsFrame.cpp
"frameverifytree" - layout/generic/nsFrame.cpp
"gc" - nsprpub/pr/src/misc/prinit.c:
"gizmo" - nsprpub/pr/include/prlog.h:**
"gnomevfs" - extensions/gnomevfs/nsGnomeVFSProtocolHandler.cpp
"htmlcontentsink" - content/html/document/src/nsHTMLContentSink.cpp
"imgRequest" - modules/libpr0n/src/imgRequest.cppPRLogModuleInfo
"io" - nsprpub/pr/src/misc/prinit.c:
"ldap" - directory/xpcom/base/src/nsLDAPProtocolModule.cpp
"ldapautocomplete" - mailnews/addrbook/src/nsLDAPAutoCompleteSession.cpp
"ldapautocomplete" - mailnews/addrbook/src/nsLDAPAutoCompleteSession.cpp
"linker" - nsprpub/pr/src/misc/prinit.c:
"mon" - nsprpub/pr/src/misc/prinit.c:
"mozStorage" - storage/src/mozStorageConnection.cpp
"msgc" - nsprpub/lib/msgc/src/prmsgc.c:
"msgcompose" - mailnews/compose/src/nsMsgComposeService.cpp
"negotiateauth" - extensions/auth/nsAuthFactory.cpp
"nsAbOutlookCardLog" - mailnews/addrbook/src/nsAbOutlookCard.cpp
"nsAbOutlookDirFactoryLog" - mailnews/addrbook/src/nsAbOutlookDirFactory.cpp
"nsAbOutlookDirectoryLog" - mailnews/addrbook/src/nsAbOutlookDirectory.cpp
"nsAbWinHelperLog" - mailnews/addrbook/src/nsAbWinHelper.cpp
"nsCSSLoader" - layout/style/nsCSSLoader.cpp
"nsCachedChromeChannel" - chrome/src/nsChromeProtocolHandler.cpp
"nsCachedChromeChannel" - rdf/chrome/src/nsChromeProtocolHandler.cpp
"nsComponentManager" - xpcom/components/nsComponentManager.cpp
"nsContentPolicy" - content/base/src/nsContentPolicy.cpp
"nsDebug" - xpcom/base/nsDebugImpl.cpp
"nsDeviceContextPS" - gfx/src/ps/nsDeviceContextPS.cpp
"nsDeviceContextXp" - gfx/src/xprint/nsDeviceContextXP.cpp
"nsDirectoryIndexStream" - netwerk/base/src/nsDirectoryIndexStream.cpp
"nsDocShell" - docshell/base/nsDocShell.cpp
"nsDocShellLeak" - docshell/base/nsDocShell.cpp
"nsDragService" - widget/src/beos/nsDragService.cpp
"nsDragService" - widget/src/gtk/nsDragService.cpp
"nsDragService" - widget/src/gtk2/nsDragService.cpp
"nsElementMap" - content/xul/document/src/nsElementMap.cpp
"nsEventQueue" - xpcom/threads/nsEventQueue.cpp
"nsFTPDirListingConv" - netwerk/streamconv/converters/nsFTPDirListingConv.cpp
"nsFtp" - netwerk/protocol/ftp/src/nsFtpProtocolHandler.cpp
"nsHostResolver" - netwerk/dns/src/nsHostResolver.cpp
"nsHttp" - netwerk/protocol/http/src/nsHttpHandler.cpp
"nsIIdleService" - widget/src/gtk2/nsIdleServiceGTK.cpp
"nsIOThreadPool" - netwerk/base/src/nsIOThreadPool.cpp
"nsJarProtocol" - modules/libjar/nsJARChannel.cpp
"nsMapiAddressBookLog" - mailnews/addrbook/src/nsMapiAddressBook.cpp
"nsMetricsService" - extensions/metrics/src/nsMetricsService.cpp
"nsNativeModuleLoader" - xpcom/components/nsNativeComponentLoader.cpp
"nsObjectFrame" - layout/generic/nsObjectFrame.cpp
"nsPipe" - xpcom/io/nsPipe3.cpp
"nsPostScriptObj" - gfx/src/ps/nsPostScriptObj.cpp
"nsPrefetch" - uriloader/prefetch/nsPrefetchService.cpp
"nsRDFContentSink" - rdf/base/src/nsRDFContentSink.cpp
"nsRDFService" - rdf/base/src/nsRDFService.cpp
"nsRDFXMLDataSource" - rdf/base/src/nsRDFXMLDataSource.cpp
"nsRequestObserverProxy" - netwerk/base/src/nsRequestObserverProxy.cpp
"nsResProtocol" - netwerk/protocol/res/src/nsResProtocolHandler.cpp
"nsSecureBrowserUI" - security/manager/boot/src/nsSecureBrowserUIImpl.cpp
"nsSharedPrefHandler" - modules/libpref/src/nsSharedPrefHandler.cpp
"nsSocketTransport" - netwerk/base/src/nsSocketTransportService2.cpp
"nsStandardURL" - netwerk/base/src/nsStandardURL.cpp
"nsStorageStream" - xpcom/io/nsStorageStream.cpp
"nsStreamCopier" - netwerk/base/src/nsAsyncStreamCopier.cpp
"nsStreamPump" - netwerk/base/src/nsInputStreamPump.cpp
"nsThread" - xpcom/threads/nsThread.cpp
"nsThreadPool" - xpcom/threads/nsThreadPool.cpp
"nsTimerImpl" - xpcom/threads/nsTimerImpl.h:
"nsWabAddressBookLog" - mailnews/addrbook/src/nsWabAddressBook.cpp
"nsWyciwygChannel" - content/html/document/src/nsWyciwygProtocolHandler.cpp
"nsXPrintContext" - gfx/src/xprint/nsXPrintContext.cpp
"nsXULCommandDispatcher" - content/xul/document/src/nsXULCommandDispatcher.cpp
"nsXULContentSink" - content/xul/document/src/nsXULContentSink.cpp
"nsXULDocument" - content/xul/document/src/nsXULDocument.cpp
"nsXULTemplateBuilder" - content/xul/templates/src/nsXULTemplateBuilder.cpp
"nscontentsink" - content/base/src/nsContentSink.cpp
"nss_cache" - security/nss/lib/pki/tdcache.c:
"nss_mod_log" - security/nss/lib/pk11wrap/debug_module.c:
"nssilock" - security/nss/lib/util/nssilock.c:
"objlc" - content/base/src/nsObjectLoadingContent.cpp
"pipnss" - security/manager/ssl/src/nsNSSComponent.cpp
"printing" - layout/base/nsDocumentViewer.cpp
"printing" - layout/printing/nsPrintData.cpp
"printing" - layout/printing/nsPrintEngine.cpp
"printing-gfx" - gfx/src/windows/nsDeviceContextWin.cpp
"printing-layout" - layout/generic/nsSimplePageSequence.cppPRLogModuleInfo
"printing-widget" - widget/src/windows/nsDeviceContextSpecWin.cppPRLogModuleInfo
"proxy" - netwerk/base/src/nsProtocolProxyService.cpp
"sched" - nsprpub/pr/src/misc/prinit.c:
"schemaValidation" - extensions/schema-validation/src/nsSchemaValidator.cppPRLogModuleInfo
"schemaValidation" - extensions/schema-validation/src/nsSchemaValidatorUtils.cppPRLogModuleInfo
"shm" - nsprpub/pr/src/misc/prinit.c:
"shma" - nsprpub/pr/src/misc/prinit.c:
"styleverifytree" - layout/generic/nsFrame.cpp
"test" - nsprpub/pr/tests/y2k.c:
"testcase" - nsprpub/lib/tests/arena.c:
"testcase" - nsprpub/pr/tests/rmdir.c:
"thebesGfx" - gfx/src/thebes/nsThebesDeviceContext.cpp
"thread" - nsprpub/pr/src/misc/prinit.c:
"timing" - modules/libutil/public/stopwatch.h:
"trace" - nsprpub/pr/src/misc/prtrace.c:
"updatedriver" - toolkit/xre/nsUpdateDriver.cpp
"webdav" - extensions/webdav/src/nsWebDAVService.cpp
"webshell" - docshell/base/nsWebShell.cpp
"winfonts" - gfx/thebes/src/gfxWindowsFonts.cpp
"xpath" - content/xslt/src/base/txLog.h:
"xpclog" - js/src/xpconnect/src/xpclog.cpp
"xpcomproxy" - xpcom/proxy/src/nsProxyObjectManager.cppPRLogModuleInfo
"xslt" - content/xslt/src/base/txLog.h:

Creating Your First Reftest, and Your Second and Third

I have updated a page on devmo and I think it is a rather nice page, though I am not objective here.

http://developer.mozilla.org/en/docs/Creating_reftest-based_unit_tests

This describes in very practical terms how to create reftests. It turns out to be way easier than it seemed at first. Really, I just had to map what reftest did into my head. Not being a long-time Mozilla-ish person, it probably took me longer than it would take others.

Here is a slice from the page, giving instructions on just the first test. Please see the page for more info. There are details in the full instructions that may be useful. And if something seems wrong, please feel free to update the page. Send me an e-mail if you would like, but wiki is more wiki if people really edit what they see. And now for the show.

---

Step 1: Build a copy of Firefox for yourself.

Step 2: Open a terminal window. Create a directory and make that your current directory.

Step 3: Create a file named foo.html with the following:

<html><head><title>reftest0001</title><body><strong>Hello!</strong></body></html>

Step 4: Create a file named bar.html with the following:

<html><head><title>reftest0001</title><body><b>Hello!</b></body></html>

Step 5: Create a file named reftest.list with the following:

== foo.html bar.html

One is now ready to run the test. Here is how I run this. Adapt this to your own platform.

% /bin/sh
$ /Users/ray/mo/browser/mozilla/dist/MinefieldDebug.app/Contents/MacOS/firefox -P minefield1 -reftest ./reftest.list 2>&1 | grep REFTEST
REFTEST PASS: file:///Users/ray/moz/reftest0001.html
$

Congratulations! You have just created your first retest!

Quick and Easy Leaks Detection on Mac OS X

If you are using a Mac OS X machine and have Apple's Developer Tools, such as Xcode, on it, you have access to a very quick and easy way to check for memory leaks. Not everyone knows about this, though it has been available for at least a few versions of the Dev Tools, so I thought I would put this out.

If an application is running and you want to see if it is leaking memory, you can do run leaks and give it either a pid as a parameter, or a string matchable to the running process, such as "firefox-bin" or "thunderbird-bin". The string matching only works if only one such process is running. Here is an example:

$ leaks firefox
Found process 260 (firefox-bin) from partial name firefox
Process 260: 1263939 nodes malloced for 147066 KB
Process 260: 76194 leaks for 7262704 total leaked bytes.
Leak: 0x19aeea00  size=4096
      0x0d0a0d30 0x6320740a 0x7373616c 0x7462223d
      0x7420226e 0x3d657079 0x62757322 0x2274696d
      0x6c617620 0x223d6575 0x646e6946 0x64692022
      0x6966223d 0x625f646e 0x6f74746f 0x3c3e226d
      0x726f662f 0x2f3c3e6d 0x0a3e696c 0x3c20200a
      0x3c3e696c 0x6e617073 0x616c6320 0x223d7373
      ...
Leak: 0x19968200  size=1024
      0x6d6c6f63 0x68697074 0x00000008 0x00000000
      0x43fa8000 0x43ba0000 0x00000000 0x00000000
      0x6b6d6f64 0x6d61676e 0x00000004 0x00000000
      0x00000000 0x00000000 0x00000000 0x00000000
      0x77696e64 0x77696e64 0x00000004 0x00000000
      0x776d6f75 0x68697074 0x00000008 0x00000000
      0x43f60000 0x43aa8000 0x00000000 0x00000000
      ...
Leak: 0x20b1a200  size=1024
      0x6d6c6f63 0x68697074 0x00000008 0x00000000

This one actually goes on for about 350,000 lines. I had been running this instance of Firefox 2.0.0.1 for about an hour, but with a lot of tab use.

You can get stack frames and symbols in the stacks of leaked objects if you set environment variables that are explained in the man page.

I think there is a lot of room for improvement in controlling memory leaks.

There is actually much more I could say about this, but that is for another discussion. Really, I just wanted to say, here is a tool. It gives information. It is pretty accurate. If anyone has questions about using it, check the man page for the leaks executable, or feel free to ask me.