internet marketing

internet marketing

internet marketing

internet marketing

Internet Marketing, Search Engine Optimization (SEO), Search Engine Marketing (SEM) & Pay Per Click Management Services.

internet marketing  Home

  internet marketing
internet marketing


internet marketing Effective Internet 

internet marketing


internet marketing Web Page Analysis

internet marketing


internet marketing Search Engine Optimization Process

internet marketing


internet marketing Competitor Web Site Analysis

internet marketing


internet marketing Internet Marketing Articles

internet marketing


internet marketing Search Engine Submission Services

internet marketing


internet marketing Real Surfing... sorta'

internet marketing Our Purpose

internet marketing

internet marketing



internet marketing

What’s in The Sniffer Log File? How to understand the raw data.

Or, more accurately, if you have a web site that produces, or you expect to produce money, you need these logs, not hit counters or other log programs.


What’s in The Sniffer Log File? How to understand the raw data.

Or, more accurately, if you have a web site that produces, or you expect to produce money, you need these logs, not hit counters or other log programs.

The sniffers produce an industry-standard log file (Extended Common Log File Format) that can be analyzed by most commercial and shareware analysis programs. This format was designed to be machine-readable, but die-hard (and curious) webmasters can learn a lot by reading the log files themselves.

A log file consists of a series of entries, one for each page that is viewed by a visitor.

A typical entry in the log file looks like this (word-wrapped to fit on this screen): - - [2/Sep/1998:19:54:14 +0000] "GET /html/win95_updates.htm HTTP/1.0" 200 54 " 2&col=New+Search&oq=%22service+release+2%22&sv=N4&lk=ip-nofra mes&nh=10" "Mozilla/4.01 [en] (Win95; I)"

Let’s break this mumbo-jumbo down into its components:

  1. The “internet name” of the user accessing your site:
    Typically, this name doesn’t identify “who” is accessing your site (nor their email address), just the domain name of their ISP. In this case, it is a user dialing in through  

    If you’re interested in who owns the name, you can always look it up in the InterNIC database. (Try it yourself) Note that InterNIC only administers .COM, .ORG and .NET domain names.

  2. Some unused fields (in our case):
    - -
    These fields sometimes show the login ID of users within password-protected parts of your site. The sniffers always leave these fields blank.

  3. The date and time of the page request (in GMT):
    [2/Sep/1998:19:54:14 +0000]
    Because your visitors access your site from all over the world, all times are recorded in Greenwich Mean Time (GMT). To convert GMT to your local time zone, use the following chart


Pacific Standard Time subtract 8 hours
Pacific Daylight Time subtract 7 hours
Eastern Standard Time subtract 5 hours
Moscow add 3 hours

Because our log files use GMT, they are perfect for any web site, no matter where your visitors are located.

  1. The name of the page viewed on your site:
    "GET /html/win95_updates.htm HTTP/1.0"
    The meat is in the middle (as they say). Just ignore anything that doesn’t look like a URL.
  2. Two more unused fields (in our case) (for the curious: Status Code and Transfer Size):
    200 54
    All the entries in your log will have these same values.
  3. The referer field—perhaps the most important information you can gather:
    " 2&col=New+Search&oq=%22service+release+2%22&sv=N4&lk=ip-nofra mes&nh=10"
    This tells you what page the visitor came from. In this case, the user was searching for the phrase “OEM service release” in InfoSeek.

    You can view the page that the user came from by typing the referer value into your browser (Try it!).
    If the user typed in your URL directly, or called it up as a bookmark (great news!) then this field will be “(none)”.
  4. The User Agent field (aka The Browser field):
    "Mozilla/4.06 [en] (Win95; I)"
    This shows what browser the user was using. In this case, it’s Netscape (code-named Mozilla) version 4.06, english, International version under Windows95.

Whew! That’s a lot of useful information, especially when you’re considering adding “advanced content” that may not be supported by all browsers to your sites (frames, cascading style sheets, Java applets...).

So how do we use all this information?

Let’s take a look at a couple of ways that we can use this information to recreate how a user used our site.

Scenario 1:

a) A visitor arrives from Lycos (where s/he searched for "best page") - - [10/Sep/1998:07:38:55 +0000] "GET /html/best_of_the_www.htm HTTP/1.0" 200 54 " klink=217&maxhits=10" "Mozilla/3.0Gold (Win95; I)"

b) drills down to the Win95 Updates part of our site (taking 7-1/2 minutes to reach it—must be taking the time to carefully read each page) - - [10/Sep/1998:07:39:56 +0000] "GET /html/web_tools.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)" - - [10/Sep/1998:07:41:13 +0000] "GET /html/conferencing.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)" - - [10/Sep/1998:07:42:04 +0000] "GET /html/web_tools.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)" - - [10/Sep/1998:07:45:31 +0000] "GET /html/web_tools.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)" - - [10/Sep/1998:07:45:50 +0000] "GET /html/conferencing.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)" - - [10/Sep/1998:07:46:19 +0000] "GET /html/win95_updates.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)"

c) then follows our link to a page in the Microsoft site (a download page) - - [10/Sep/1998:07:47:59 +0000] "GET HTTP/1.0" 200 54 "/html/win95_updates.htm" "Mozilla/3.0Gold (Win95; I)"

d) returns 3-1/2 minutes later (yay!) by using the Back Arrow
NOTE: No other web tracking tool will show you this information! Not even standard web server logs. - - [10/Sep/1998:07:51:17 +0000] "GET /html/win95_updates.htm HTTP/1.0" 200 54 "" "Mozilla/3.0Gold (Win95; I)"

e) then again follows a link off our site to get the 12+ Win95 updates from Microsoft - - [10/Sep/1998:07:52:08 +0000] "GET HTTP/1.0" 200 54 "/html/win95_updates.htm" "Mozilla/3.0Gold (Win95; I)"

How to Interpret Scenario 1

These log file entries tell us a lot of valuable information about how well our site is (or isn’t) designed:

This visitor took 7-1/2 minutes to read 6 pages (over a minute per page). It appears that the visitor is taking time to read each page thoroughly, rather than just randomly clicking through the site.The visitor came back after clicking an off-site link! We find that this is one of the strongest indicators of good site design.

We know of no other tool that can show you this.

It is completely unaffected by browser caching and proxy servers. You can completely track your visitors’ sessions, whether they use the Back Arrow or not.

By analyzing hundreds of users’ visits with a commercial or shareware log analysis program, you can find out what is and what isn’t working with your site’s design.

It’s just like standing in your shop while shoppers wander in and out of your store.

Scenario 2:

I recently posted an article to the link exchange newsgroup discussing the popularity of various search engines.

Many web marketers call newsgroup postings one of the most effective marketing tools available today.

If I were using traditional web server logs, I would have absolutely no idea of how many people were actually reading my newsgroup messages.

Standard web server logs cannot capture this valuable marketing information.

However, by simply dropping a sniffer in my newsgroup messages, I now know that >75 people read this post this first day--a lot more eyeballs than saw my postings on other newsgroups.

With this sniffer, I can gauge:

a) The number of eyeballs reading my postings, and

b) The percentage click-throughs as a result. i.e. no more guessing which newsgroups are truly effective at reaching your target market.

Here’s what my logs show: the 2 most recent people to read my posting were: - - [10/Sep/1998:20:14:10 +0000] "GET news:// m HTTP/1.0" 200 54 "le.discuss.popularity" "Mozilla/3.01Gold (Win95; I)" - - [10/Sep/1998:20:59:27 +0000] "GET news:// m HTTP/1.0" 200 54 "le.discuss.popularity" "Mozilla/3.01Gold (Win95; I)"

Notice that they didn’t need to click through to our site to be logged—they only needed to read an article we posted on a newsgroup.

Armed with this information, I can now track:

newsgroup viewership, and click-through rates.
No more second-guessing why your web traffic increased.

Which newsgroups are generating the best leads for your business?
Which newsgroups are not being read?
Which signature slogans have the highest appeal?
We know of no other tool that can reliably tell you this information.



915 Neptune Encinitas, CA 92024
Contact Us
Internet marketing & online web promotion for business web sites.
All rights reserved. Copyright © 1996-

internet marketing