Overview of TCP/IP Troubleshooting Tools

Table 3.1 lists the diagnostic utilities included with Microsoft TCP/IP; they are described in more detail in the following pages. All are useful to identify and resolve TCP/IP networking problems.

For a quick reference chart of these TCP/IP tools, as well as remote administration tools, see the appendix "TCP/IP Remote Utilities" in this book.

These tools are discussed in their own chapters of the Windows 2000 Resource Kit.

Arp

Arp allows you to view and modify the ARP cache. If two hosts on the same subnet cannot ping each other successfully, try running the arp -a command on each computer to see whether the computers have the correct media access control (MAC) addresses listed for each other. You can use Ipconfig to determine a host’s correct MAC address.

You can also use Arp to view the contents of the ARP cache by typing arp -a at a command prompt. This displays a list of the ARP cache entries, including their MAC addresses. Following is an example list of ARP cache entries.

C:\>arp -a

Interface: 172.16.0.142 on Interface 0x2

Internet address Physical Address Type

172.16.0.1 00-e0-34-c0-a1-40 dynamic

172.16.1.231 00-00-f8-03-6d-65 dynamic

172.16.3.34 08-00-09-dc-82-4a dynamic

172.16.4.53 00-c0-4f-79-49-2b dynamic

172.16.5.102 00-00-f8-03-6c-30 dynamic

If another host with a duplicate IP address exists on the network, the ARP cache might have the MAC address for the other computer placed in it, and this can lead to intermittent problems with address resolution. When a computer on the local network sends an ARP Request to resolve the address, it forwards its data to the MAC address corresponding to the first ARP Reply it receives. Arp can help by listing, adding, and removing the relevant entries.

You can use arp -d <IP address> to delete incorrect entries. Use arp -s <MAC address> (where the MAC address is formatted as hexadecimal bytes separated by dashes) to add new static entries; these static entries do not expire from the ARP cache. However, static entries do not persist after a reboot. For persistent static ARP cache entries, you must create a batch file run from the Startup group.

Use arp -N <IP address> to list all the ARP entries for the network interface specified by <IP address>. Table 3.2 lists all Arp switches.

Hostname

Hostname displays the name of the host on which the command is issued. The command has no other switches or parameters. The host name displayed matches the name configured on the Network Identification table in Control Panel-System.

Ipconfig

IPConfig is a command-line tool that displays the current configuration of the installed IP stack on a networked computer.

When used with the /all switch, it displays a detailed configuration report for all interfaces, including any configured WAN miniports (typically used for remote access or VPN connections). Output can be redirected to a file and pasted into other documents. A sample report is shown here:

C:>\ipconfig /all

Windows 2000 IP Configuration

Host Name . . . . . . . . . . . . : TESTPC1

Primary DNS Suffix . . . . . . . : reskit.com

Node Type . . . . . . . . . . . . : Hybrid

IP Routing Enabled. . . . . . . . : No

WINS Proxy Enabled. . . . . . . . : No

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : dns.reskit.com

Description . . . . . . . . . . . : Acme XL 10/100Mb Ethernet NIC

Physical Address. . . . . . . . . : 00-CC-44-79-C3-AA

DHCP Enabled. . . . . . . . . . . : Yes

IP Address. . . . . . . . . . . . : 172.16.245.111

Subnet Mask . . . . . . . . . . . : 255.255.248.0

Default Gateway . . . . . . . . . : 172.16.240.1

DHCP Server . . . . . . . . . . . : 172.16.248.8

DNS Servers . . . . . . . . . . . : 172.16.55.85

172.16.55.134

172.16.55.54

Primary WINS Server . . . . . . . : 172.16.248.10

Secondary WINS Server . . . . . . : 172.16.248.9

Lease Obtained. . . . . . . . . . : Friday, May 05, 1999 2:21:40 PM

Lease Expires . . . . . . . . . . : Monday, May 07, 1999 2:21:40 PM

A number of other useful parameters for Ipconfig include /flushdns, which deletes the DNS name cache; /registerdns, which refreshes all DHCP leases and re-registers DNS names; and /displaydns which displays the contents of the DNS resolver cache.

The /release <adapter> and /renew <adapter> options release and renew the DHCP-allocated IP address for a specified adapter. If no adapter name is specified, the DHCP leases for all adapters bound to TCP/IP are released or renewed.

For /setclassid, if no class ID is specified, then the Class ID is removed. Table 3.3 lists all Ipconfig switches.

The /showclassid and /setclassid options allow you to manipulate user class IDs from the command line. The user class IDs are options that a system administrator may set on the DHCP server to configure a client computer to identify itself with the server. Issuing the command ipconfig /showclassid <adapter> sends a query to the client’s server; the server responds by providing the available classes. Once you know which classes are available, you can issue a command like ipconfig /setdhcpclassid <adapter> <class ID to set on the server> to set the class ID that the client will use from that point on. For more information about DHCP and class IDs, see "Dynamic Host Configuration Protocol" in this book.

Nbtstat

Nbtstat is designed to help troubleshoot NetBIOS name resolution problems. When a network is functioning normally, NetBIOS over TCP/IP (NetBT) resolves NetBIOS names to IP addresses. It does this through several options for NetBIOS name resolution, including local cache lookup, WINS server query, broadcast, LMHOSTS lookup, Hosts lookup, and DNS server query.

The nbtstat command removes and corrects preloaded entries using a number of case-sensitive switches. The nbtstat -a <name> command performs a NetBIOS adapter status command on the computer name specified by <name>. The adapter status command returns the local NetBIOS name table for that computer as well as the MAC address of the adapter card. The nbtstat -A <IP address> command performs the same function using a target IP address rather than a name.

The nbtstat -c option shows the contents of the NetBIOS name cache, which contains NetBIOS name-to-IP address mappings.

nbtstat -n displays the names that have been registered locally on the system by NetBIOS applications such as the server and redirector.

The nbtstat -r command displays the count of all NetBIOS names resolved by broadcast and by querying a WINS server. The nbtstat -R command purges the name cache and reloads all #PRE entries from the LMHOSTS file. #PRE entries are the LMHOSTS name entries that are preloaded into the cache. For more information about the LMHOSTS file, see the appendix "LMHOSTS" in this book.

Nbtstat -RR sends name release packets to the WINS server and starts a refresh, thus re-registering all names with the name server without having to reboot. This is a new option in Windows NT 4.0 with Service Pack 4 as well as in Windows 2000.

You can use nbtstat -S to list the current NetBIOS sessions and their status, including statistics. Sample output looks like this:

C:\>nbtstat -S

Local Area Connection:

NetBIOS Connection Table

Local Name State In/Out Remote Host Input Output

------------------------------------------------------------------

TESTPC1 <00> Connected Out 172.16.210.25 6MB 5MB

TESTPC1 <00> Connected Out 172.16.3.1 108KB 116KB

TESTPC1 <00> Connected Out 172.16.3.20 299KB 19KB

TESTPC1 <00> Connected Out 172.16.3.4 324KB 19KB

TESTPC1 <03> Listening

Finally, nbtstat -s provides a similar set of session listings, but provides the remote computer names, rather than their IP addresses.

Note The options for the Nbtstat command are case sensitive.

The Nbtstat switches are listed in Table 3.4.

Netdiag

Netdiag is a utility that helps isolate networking and connectivity problems by performing a series of tests to determine the state of your network client and whether it is functional. These tests and the key network status information they expose give network administrators and support personnel a more direct means of identifying and isolating network problems. Moreover, because this tool does not require parameters or switches to be specified, support personnel and network administrators can focus on analyzing the output, rather than training users about tool usage.

Netdiag diagnoses network problems by checking all aspects of a host computer’s network configuration and connections. Beyond troubleshooting TCP/IP issues, it also examines a host computer’s Internetwork Packet Exchange (IPX) and NetWare configurations.

Run Netdiag whenever a computer is having network problems. The utility tries to diagnose the problem and can even flag problem areas for closer inspection. It can fix simple DNS problems with the optional /fix switch.

For more information about Netdiag, see Windows 2000 Support Tools Help. For information about installing and using the Windows 2000 Support Tools and Support Tools Help, see the file Sreadme.doc in the \Support\Tools folder of the Windows 2000 operating system CD.

Netdiag performs its tests by examining .dll files, output from other tools, and the system registry to find potential problem spots. It checks to see which network services or functions are enabled and then runs the network configuration tests listed in Table 3.5, in the order presented. If a computer is not running one of the services listed, the test is skipped.

The required syntax for Netdiag is simple. The tool can be configured to perform any subset of its exhaustive list of tests by careful use of the /test or /skip options.

Although no parameters or syntax need be specified, several options are available for Netdiag, primarily to increase or decrease the level of detail in its reports. These switches are shown in the Table 3.6. Complete details on the /test and /skip options can be found by typing netdiag /? at a command prompt; this returns a complete list of more than 20 tests that can be singled out or skipped.

In general, Netdiag calls Ipconfig and returns a structure that contains most of the general information that ipconfig /all prints. It takes that information from the registry and by calling the various drivers.

Netdiag prints the string [FATAL] when it detects a condition that needs to be fixed immediately. By contrast, the string [WARNING] signals a failure condition that can be put off for a while.

Netstat

Netstat displays protocol statistics and current TCP/IP connections. From a command prompt, type Netstat -a to display all connections and listening ports. Type netstat -r to display the contents of the IP routing table and any persistent routes. The -n switch tells Netstat not to convert addresses and port numbers to names, which speeds up execution. The netstat -s option shows all protocol statistics. The netstat -p <protocol> option can be used to show statistics for a specific protocol or together with the -s option to show connections only for the protocol specified. The -e switch displays interface statistics. Sample output for the netstat -e command is shown here:

C:\>netstat -e

Interface Statistics

Received Sent

Bytes 372959625 123567086

Unicast packets 134302 145204

Non-unicast packets 55937 886

Discards 0 0

Errors 0 0

Unknown protocols 1757381

Discards are the packets received that contained errors or could not be processed. Errors indicate packets that are damaged, including packets sent by the local computer that were damaged while in the buffer.

Both of these types of errors should be at or near zero. If not, errors in the Sent column indicate that the local network might be overloaded or that there might be a bad physical connection between the local host and the network. High errors and discards in the Receive column indicate an overloaded local net, an overloaded local host, or a physical problem with the network.

The following output shows a sample report for the netstat -a -n command.

C:\>netstat -a -n

Active Connections

Proto Local Address Foreign Address State

TCP 0.0.0.0:42 0.0.0.0:0 LISTENING

TCP 0.0.0.0:88 0.0.0.0:0 LISTENING

TCP 0.0.0.0:135 0.0.0.0:0 LISTENING

TCP 0.0.0.0:389 0.0.0.0:0 LISTENING

TCP 0.0.0.0:445 0.0.0.0:0 LISTENING

TCP 0.0.0.0:593 0.0.0.0:0 LISTENING

TCP 0.0.0.0:1038 0.0.0.0:0 LISTENING

TCP 0.0.0.0:1041 0.0.0.0:0 LISTENING

TCP 0.0.0.0:1048 0.0.0.0:0 LISTENING

TCP 0.0.0.0:1723 0.0.0.0:0 LISTENING

TCP 0.0.0.0:3268 0.0.0.0:0 LISTENING

TCP 10.99.99.1:53 0.0.0.0:0 LISTENING

TCP 10.99.99.1:139 0.0.0.0:0 LISTENING

TCP 10.99.99.1:389 10.99.99.1:1092 ESTABLISHED

TCP 10.99.99.1:1092 10.99.99.1:389 ESTABLISHED

TCP 10.99.99.1:3604 10.99.99.1:135 TIME_WAIT

TCP 10.99.99.1:3605 10.99.99.1:1077 TIME_WAIT

UDP 0.0.0.0:135 *:*

UDP 0.0.0.0:445 *:*

UDP 0.0.0.0:1087 *:*

UDP 10.99.99.1:53 *:*

UDP 10.99.99.1:137 *:*

UDP 10.99.99.1:138 *:*

The number after the colon indicates which port number each connection is using. For a complete port reference list, see the appendix "TCP and UDP Port Assignments" in this book.

The following output shows the TCP, IP, ICMP, and UDP statistics for the local host.

D:\>netstat -s

IP Statistics

Packets Received = 3175996

Received Header Errors = 0

Received Address Errors = 38054

Datagrams Forwarded = 0

Unknown Protocols Received = 0

Received Packets Discarded = 0

Received Packets Delivered = 3142564

Output Requests = 3523906

Routing Discards = 0

Discarded Output Packets = 0

Output Packet No Route = 0

Reassembly Required = 0

Reassembly Successful = 0

Reassembly Failures = 0

Datagrams Successfully Fragmented = 0

Datagrams Failing Fragmentation = 0

Fragments Created = 0

ICMP Statistics

Received Sent

Messages 462 33

Errors 0 0

Destination Unreachable 392 4

Time Exceeded 0 0

Parameter Problems 0 0

Source Quenchs 0 0

Redirects 0 0

Echos 1 22

Echo Replies 12 1

Timestamps 0 0

Timestamp Replies 0 0

Address Masks 0 0

Address Mask Replies 0 0

TCP Statistics

Active Opens = 12164

Passive Opens = 12

Failed Connection Attempts = 79

Reset Connections = 11923

Current Connections = 1

Segments Received = 2970519

Segments Sent = 3505992

Segments Retransmitted = 18

UDP Statistics

Datagrams Received = 155620

No Ports = 16578

Receive Errors = 0

Datagrams Sent = 17822

Table 3.7 summarizes the switches available for use with Netstat.

Nslookup

Nslookup is a useful tool for troubleshooting DNS problems, such as host name resolution. When you start Nslookup, it shows the host name and IP address of the DNS server that is configured for the local system, and then display a command prompt for further queries. If you type a question mark (?), Nslookup shows all available commands. You can exit the program by typing exit.

To look up a host’s IP address using DNS, type the host name and press Enter. Nslookup defaults to using the DNS server configured for the computer on which it is running, but you can focus it on a different DNS server by typing server <name> (where <name> is the host name of the server you want to use for future lookups). Once another server is specified, anything entered after that point is interpreted as a host name.

Nslookup employs the domain name devolution method. If you type in a host name and press ENTER, Nslookup appends the domain suffix of the computer (such as cswatcp.reskit.com) to the host name before querying the DNS. If the name is not found, then the domain suffix is "devolved" by one level (in this case to reskit.com) and the query is repeated. Windows 2000 computers only devolve names to the second level domain (reskit.com in this example), so if this query fails, no further attempts are made to resolve the name. If a fully qualified domain name is typed in (as indicated by a trailing dot) then the DNS server is only queried for that name and no devolution is performed. To look up a host name that is completely outside your domain, you must type in a fully qualified domain name.

Nslookup’s debug mode is a useful troubleshooting feature; you can set the local computer into this mode by typing set debug, or for even greater detail, set d2.

In debug mode, Nslookup lists the steps being taken to complete its commands, as shown in this example:

C:\>nslookup

(null) testpc1.reskit.com

Address: 172.16.8.190

> set d2

> rain-city

(null) testpc1.reskit.com

Address: 172.16.8.190

------------

SendRequest(), len 49

HEADER:

opcode = QUERY, id = 2, rcode = NOERROR

header flags: query, want recursion

questions = 1, answers = 0, authority records = 0, additional = 0

QUESTIONS:

rain-city.reskit.com, type = A, class = IN

------------

------------

Got answer (108 bytes):

HEADER:

opcode = QUERY, id = 2, rcode = NOERROR

header flags: response, auth. answer, want recursion, recursion avail.

questions = 1, answers = 2, authority records = 0, additional = 0

QUESTIONS:

rain-city.reskit.com, type = A, class = IN

ANSWERS:

-> rain-city.reskit.com

type = CNAME, class = IN, dlen = 31

canonical name = seattle.reskit.com

ttl = 86400 (1 day)

-> seattle.reskit.com

type = A, class = IN, dlen = 4

internet address = 172.16.2.3

ttl = 86400 (1 day)

------------

(null) seattle.reskit.com

Address: 172.16.2.3

Aliases: rain-city.reskit.com

In this example, the user issued the set d2 command to set Nslookup to debug mode, then the user tried a simple address lookup for the host name "rain-city." The first two lines of output show the host name and IP address of the DNS server where the lookup was sent. As the next paragraph shows, the domain suffix of the local computer (reskit.com) was appended to the name "rain-city," and Nslookup submitted this question to the DNS server.

The next paragraph in the example indicates that Nslookup received an answer from the DNS server. The DNS server provided two answer records in response to one question. The question is repeated in the response, along with the two answer records. In this case, the first answer record indicates that the name "rain-city.reskit.com" is actually a cname, or canonical name (alias) for the host name "seattle.reskit.com." The second answer record lists the IP address for that host as 172.16.2.3.

Table 3.8 summarizes all Nslookup switches. Identifiers are shown in upper case, and optional commands are shown in brackets.

PathPing

The PathPing tool is a route tracing tool that combines features of Ping and Tracert with additional information that neither of those tools provides. PathPing sends packets to each router on the way to a final destination over a period of time, and then computes results based on the packets returned from each hop. Since PathPing shows the degree of packet loss at any given router or link, you can pinpoint which routers or links might be causing network problems. A number of switches are available, as shown in Table 3.9.

The default number of hops is 30, and the default wait time before a time-out is three seconds (3000 milliseconds). The default period is 250 milliseconds, and the default number of queries to each router along the path is 100.

Below is a typical PathPing report. Note that the compiled statistics that follow the hop list indicate packet loss at each individual router.

D:\>pathping -n testpc1

Tracing route to testpc1 [7.54.1.196]

over a maximum of 30 hops:

0 172.16.87.35

1 172.16.87.218

2 192.68.52.1

3 192.68.80.1

4 7.54.247.14

5 7.54.1.196

Computing statistics for 125 seconds...

Source to Here This Node/Link

Hop RTT Lost/Sent = Pct Lost/Sent = Pct Address

0 172.16.87.35

0/ 100 = 0% |

1 41ms 0/ 100 = 0% 0/ 100 = 0% 172.16.87.218

13/ 100 = 13% |

2 22ms 16/ 100 = 16% 3/ 100 = 3% 192.68.52.1

0/ 100 = 0% |

3 24ms 13/ 100 = 13% 0/ 100 = 0% 192.68.80.1

0/ 100 = 0% |

4 21ms 14/ 100 = 14% 1/ 100 = 1% 7.54.247.14

0/ 100 = 0% |

5 24ms 13/ 100 = 13% 0/ 100 = 0% 7.54.1.196

Trace complete.

When PathPing is run, the first results you see list the route as it is tested for problems. This is the same path that is shown via Tracert. PathPing then displays a busy message for the next 125 seconds (this time varies by the hop count, requiring 25 seconds per hop). During this time PathPing gathers information from all the routers previously listed and from the links between them. At the end of this period, it displays the test results.

The two rightmost columns — "This Node/Link Lost/Sent=%" and "Address" — contain the most useful information. The link between 172.16.87.218 (hop 1), and 192.68.52.1 (hop 2) is dropping 13 percent of the packets. All other links are working normally. The routers at hops 2 and 4 also drop packets addressed to them (as shown in the "This Node/Link" column), but this loss does not affect their forwarding path.

The loss rates displayed for the links (marked as a "|" in the rightmost column) indicate losses of packets being forwarded along the path. This loss indicates link congestion. The loss rates displayed for routers (indicated by their IP addresses in the rightmost column) indicate that those routers’ CPUs or packet buffers might be overloaded. These congested routers might also be a factor in end-to-end problems, especially if packets are forwarded by software routers.

The raw data that PathPing obtains describes how many ICMP Echo Requests are lost between the source and an intermediate router. Figure 3.1 shows how PathPing estimates the per-hop loss statistics. While at first this calculation might seem trivial, it is complicated by differences between the forwarding code path and the code path taken in responding to ping packets (ICMP Echo Requests/Replies).

In the diagram, the vertical lines indicate the extra processing taken when an ICMP Echo Request is sent to the local computer. This kicks it out of the fast path and delivers it to an ICMP module (often using separate queues and processors). Assuming no packets are dropped due to queue overflows, the ICMP module then generates an ICMP Echo Reply, which is forwarded back to the original sender.

Since packet loss can occur in the path indicated by the vertical lines (but such loss does not necessarily imply loss on the horizontal forwarding path itself), the raw numbers obtained from pings do not by themselves determine end-to-end packet loss. For example, pinging an intermediate router might create a 10 percent loss even though no end-to-end packet loss is occurring. PathPing’s algorithm uses the change in values from hop-to-hop to estimate actual per hop loss rather than losses in the higher-level router components. This actual per hop loss is the result provided in the "This Node/Link" column of the final PathPing report.

Ping

Ping is the primary tool for troubleshooting IP-level connectivity. Type ping -? at a command prompt to see a complete list of available command-line options. Ping allows you to specify the size of packets to use (the default is 32 bytes), how many to send, whether to record the route used, what Time To Live (TTL) value to use, and whether to set the "don’t fragment" flag.

When a ping command is issued, the utility sends an ICMP Echo Request to a destination IP address. Try pinging the IP address of the target host to see if it responds. If that succeeds, try pinging the target host using a host name. Ping first attempts to resolve the name to an address through a DNS server, then a WINS server (if one is configured), then attempts a local broadcast. When using DNS for name resolution, if the name entered is not a fully qualified domain name, the DNS name resolver appends the computer’s domain name or names to generate a fully qualified domain name.

If pinging by address succeeds but pinging by name fails, the problem usually lies in name resolution, not network connectivity. Note that name resolution might fail if you do not use a fully qualified domain name for a remote name. These requests fail because the DNS name resolver is appending the local domain suffixes to a name that resides elsewhere in the domain hierarchy.

The following example illustrates how to send two pings, each 1450 bytes in size, to address 172.16.99.2:

C:\>ping -n 2 -l 1450 172.16.99.2

Pinging 172.16.99.2 with 1450 bytes of data:

Reply from 172.16.99.2: bytes=1450 time<10ms TTL=62

Reply from 172.16.99.2: bytes=1450 time<10ms TTL=62

Ping statistics for 172.16.99.2:

Packets: Sent = 2, Received = 2, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 0ms, Maximum = 0ms, Average = 0ms

By default, Ping waits one second for each response to be returned before timing out. If the remote system being pinged is across a high-delay link such as a satellite link, responses might take longer to be returned. Use the -w (wait) switch to specify a longer time-out.

Note If Ping indicates a high packet loss or slow round-trip response on a LAN, your network might have a hardware problem. On a WAN, these results may be normal, and TCP/IP is designed to handle the variability. On a LAN, round-trip time is very low, and you see little or no packet loss. If this isn’t the case, test your cables, cable terminations, hubs, switches, and transceivers.

Table 3.10 lists Ping switches.

Route

Route is used to view and modify the IP routing table. Route Print displays a list of current routes that the host knows. Sample output from the route command is shown in "Troubleshooting IP Routing" later in this chapter. Route Add adds routes to the table. Route Delete removes routes from the host’s routing table.

Note Routes added to a routing table are not made persistent unless the -p switch is specified. Non-persistent routes only last until the computer is restarted or until the interface is deactivated. The interface can be deactivated when the plug-and-play interface is unplugged (such as for laptops and hot-swap PCs), when the wire is removed from the media card (if the adapter supports media fault sensing), or when the interface is manually disconnected from the adapter in the Network and Dial-up Connections folder.

The usage for Route is route [-f] [-p] [command [destination]] [MASK netmask] [gateway] [metric metric] [if interface].

The commands usable in the syntax above are Print, Add, Delete, and Change. Table 3.11 lists these commands as well as the other Route switches and parameters.

All symbolic names used for the destination are looked up in the network database file NETWORKS. The symbolic names for the gateway are looked up in the host name database file HOSTS. If the command is print or delete, the destination value can be a wildcard value specified by an asterisk ("*"). If the destination specified contains a * or ?, it is treated as a shell pattern, and only matching destination routes are printed. The asterisk matches any string, and the question mark matches any one character. For example, 157.*.1, 157.*, 127.*, *224* are all valid uses of the wildcard asterisk.

Using an invalid combination of a destination and netmask value generates a "route: bad gateway address netmask" error. This sort of error message appears, for example, when a bitwise logical AND between the destination and mask does not equal the destination value.

Tracert

Tracert is a route tracing utility that display a list of near-side router interfaces of the routers along the path between a source host and a destination. Tracert uses the IP TTL field in ICMP Echo Requests and ICMP Time Exceeded messages to determine the path from a source to a destination through an IP internetwork.

Note that some routers silently drop packets with expired TTLs. These routers do not appear in the Tracert display.

Tracert works by incrementing the TTL value by one for each ICMP Echo Request it sends, then waiting for an ICMP Time Exceeded message. The TTL values of the Tracert packets start with an initial value of one; the TTL of each trace after the first is incremented by one. A packet sent out by Tracert travels one hop further on each successive trip.

Figure 3.2 shows how Tracert works. Tracert is being run on Host A, and is following the path to Host B. At Router 1 and Router 2, the TTL is decremented to 0, causing each router to send an ICMP Time Exceeded message. When the ICMP Echo Request is received at Host B, it sends back an ICMP Echo Reply.

Note The UNIX version of Tracert performs the same function as the Windows version except that the IP payload is a UDP packet addressed to a (presumably) unknown destination UDP port. Intermediate routers send back ICMP Time Expired messages recording the route taken and the final destination sends back an ICMP Destination Unreachable-Port Unreachable message.

The UDP payload from the UNIX Tracert tool can cross routers and firewalls, whereas the ICMP Echo Request messages might not due to ICMP filtering. To avoid this problem in Windows 2000, turn off packet filtering as described in "Check Packet Filtering" later in this chapter, then try using Tracert again.

Following is an example of a tracert command output. Beginning with the first entry, it shows each router discovered on the way to the final destination in sequence; after the first two routers the trace reaches its destination. The lines of the tracert display have been indented for readability.

C:\tracert reskit

Tracing route to reskit.dns.microsoft.com [172.16.180.113] over a maximum of 30 hops:

1 <10 ms <10 ms <10 ms

ms28-rtr1-f10-00.network.microsoft.com [157.59.0.1]

2 <10 ms <10 ms <10 ms

ms42-rtr1-a5-00-1.network.microsoft.com [157.54.247.98]

1 <10 ms <10 ms <10 ms

RESKIT [172.16.180.113]

In cases where a trace either fails to reach its destination or no ICMP Time Exceeded messages are returned, the output shows an asterisk in each of the three time columns where the round-trip time is usually displayed, and shows a "Request timed out." or other error message in the right-hand column where a domain name or IP address is usually displayed.

Table 3.12 lists Tracert switches.

Troubleshooting Overview

When troubleshooting any problem, ask yourself the following questions:

Ideally, a review of the location and timing of the problem helps narrow the problem’s scope. In addition, you can examine TCP/IP failures systematically by referring to the steps needed for successful computer communications . These steps are described in the following sections; suggested methods for troubleshooting begin in the "Unable to Resolve a Host or NetBIOS Name" section of this chapter.

TCP/IP Communication

The TCP/IP process for two computers to communicate over a network breaks down into four distinct steps. The four steps the TCP/IP protocol takes on a sending host before sending out a packet are:

The following sections describe each portion of this process. The TCP/IP stack always follows this sequence when determining how to get a packet from point to point. To skip directly to the standard troubleshooting sequence, see the "Unable to Resolve a Host or NetBIOS Name" section of this chapter.

Resolving a Name to an IP Address

If the destination to be reached by an application is in the format of a NetBIOS name or host name, then name resolution is required before IP can send the first packet. IP only understands IP addresses; host and NetBIOS names are each resolved to an IP address in different ways.

NetBIOS names can be directly resolved to an IP address through four mechanisms: consulting the cache, broadcasting, checking the LMHOSTS file or querying a WINS server. The order in which Windows 2000 uses these mechanisms depends upon the node type of the client.

Windows 2000 always begins by checking the host computer’s internal NetBIOS name cache. If this fails to provide an IP address, the NetBIOS name can be resolved to an IP address using a broadcast, an LMHOST file, or a WINS servers. Which of these three is used first by any particular computer depends on its node type; the default node type is hybrid or H-node, which starts by querying a WINS server, then attempts a local broadcast to resolve the name. For a detailed discussion of node types, see "Windows 2000 TCP/IP" in this book. If these mechanisms are exhausted, a client queries its Host file, and failing that, queries its DNS server if it is configured to use one.

Note that if the only problem is NetBIOS name resolution, the computershould still be able to reach the remote resource by IP address. The tools used to diagnose NetBIOS name resolution problems are Nbtstat, Nslookup, and the net use command.

For more information about WINS, see "Windows Internet Name Service" in this book.

Host names can be directly resolved by the Hosts file or by a DNS server. Problems here usually involve a misconfigured Hosts file or DNS server, a misspelled Hosts file entry or IP address, or multiple entries for a single host in a Hosts file. The tools used to diagnose host or domain resolution problems are Nslookup or Netdiag.

For more information about DNS, see "Introduction to DNS" and "Windows 2000 DNS" in this book.

Determining Whether an Address Is Local or Remote

The subnet mask and the IP address are used together to determine whether a destination IP address is local or remote.

At this point, configuration mistakes, such as a misconfigured subnet mask, can lead to the host becoming unable to reach other hosts on other local subnets, though it can still communicate with remote hosts on distant networks and hosts on its own subnet.

If the address is local, delivery requires little additional effort. ARP resolves the IP address into a hardware address, typically a Media Access Control (MAC) address for the destination Ethernet card. The problems found here are typically problems with the ARP cache (such as duplicate addresses) or the subnet mask, and can be solved by using the Arp or Ipconfig tools.

If the address is remote, the next step is to determine which gateway to use to reach the remote address. In a network with only a single router acting as an external connection, the problem is relatively straightforward. However, in any network with more than one router attached, determining which gateway to use is more difficult.

IP solves the problem by consulting its routing table This routing table serves as a decision tree that enables IP to decide which interface and which gateway it should use to send the outgoing traffic. The routing table contains many individual routes; each route consists of a destination, net mask, gateway interface and metric.

If two routes are identical, the route with the lowest metric is chosen over the route with a higher metric. Note that the routing table is parsed from the most specific to the most general, so the packet is sent to the first gateway whose routing table entry matches the packet’s destination. In the case of a tie, the choice is made in round-robin fashion. Problems found here are addressed with the Route tool or with network configuration changes.

Once the correct gateway is determined, the ARP process is performed for the gateway address just as it is for any other local address. The ARP broadcast returns a hardware address, and the message is sent to the gateway to be routed further.

Unable to Reach a Host or NetBIOS Name

TCP/IP for Windows 2000 allows an application to communicate over a network with another computer by using three basic types of destination designations:

This section describes how to troubleshoot either host name or NetBIOS name resolution problems. Problems with IP addressing are covered in "Unable to Reach an IP Address" later in this chapter. Both of these issues are outlined in Figures 3.3-3.5, which provide a simplified flowchart to guide troubleshooting.

The easiest way to distinguish host name problems from NetBIOS name resolution problems is to find out whether the failing application uses NetBIOS or Sockets. If it uses Sockets, the problem lies with a DNS/host name resolution. Among the most common applications, the NetBIOS family includes the various NET commands or the Windows NT 4.0 administrator tools while Sockets and WinSockets applications include Telnet, Ftp, and web browsers.

The following sections describe the processes that occur when using a host name or a NetBIOS name to connect with hosts on a TCP/IP network.

Error 53

The most common symptom of a problem in NetBIOS name resolution is when the Ping utility returns an Error 53 message. The Error 53 message is generally returned when name resolution fails for a particular computer name. Error 53 can also occur when there is a problem establishing a NetBIOS session. To distinguish betweenthese two cases, use the following procedure:

If the computer is on the local subnet, confirm that the name is spelled correctly and that the target computer is running TCP/IP as well. If the computer is not on the local subnet, be sure that its name and IP address mapping are available in the DNS database, the Hosts or LMHOSTS file, or the WINS database.

If all TCP/IP elements appear to be installed properly, use Ping with the remote computer to be sure its TCP/IP protocol is working.

Cannot Connect to Remote Systems Using Host Name

If the problem is not NetBIOS but Sockets, the problem is related to either a Hosts file or a DNS configuration error. To determine why only IP addresses but not host names work for connections to remote computers, make sure that the appropriate Hosts file and DNS setup have been configured for the computer.

Note that this procedure does not take DHCP clients into account; these clients do not have DNS server in the list.

If you are having trouble connecting to a remote system using a host name and are using a Hosts file for name resolution, the problem may be with the contents of that file. Make sure the name of the remote computer is spelled correctly in the Hosts file and by the application using it.

The Hosts file or a DNS server is used to resolve host names to IP addresses whenever you use TCP/IP utilities such as Ping. You can find the Hosts file in \%SystemRoot%\System32\Drivers\Etc.

This file is not dynamic; all entries are made manually. The file format is the following:

172.16.48.10 testpc1 # Remarks are denoted with a #.

The IP address and friendly host name are always separated by one or more space or tab characters.

A computer using its Hosts file for name resolution performs the following steps.

The following Hosts file problems can cause networking errors:

If you are using DNS, be sure that the IP addresses of the DNS servers are correct and in the proper order. Use Ping with the remote computer’s host name and then with its IP address to determine whether the host address is being resolved properly. If the host name ping fails and the IP address ping succeeds, the problem is with name resolution. You can test whether the DNS servers are running by pinging their IP addresses or by opening a Telnet session to port 53 on the DNS server. If the connection is established successfully, the DNS service is working on the DNS server. Once you’ve verified that the DNS service is running, you can perform NSLookup queries to the DNS server to further verify the status of the records you are looking for.

If ping by IP address and by name fail, the problem is with network connectivity, such basic connectivity or routing. For more information about troubleshooting network connectivity, see "Troubleshooting IP Routing" later in this chapter.

A brief summary of how DNS resolves host names is provided in this section. For more information about DNS, see "Windows 2000 DNS" in this book.

DNS is a distributed database that maps domain names to data. A user can query DNS using hierarchical, friendly names to locate computers and other resources on an IP network. This allows it to largely replace the function once performed by the Hosts file. To do so, it resolves friendly names to IP addresses as follows (in the worst-case scenario, an answer might be provided by any server along the line, preventing the need for further iterative queries):

Note that this example is specific to the Internet. For more information about DNS host name resolution, recursive queries, and iterative queries, see "Introduction to DNS" in this book.

Errors in name resolution can occur if the entries in a DNS server or client are not configured correctly, if the DNS server is not running, or if there is a problem with network connectivity. To determine the cause of any name resolution problem, you can use the Nslookup utility.

Failed queries will return a variety of messages, depending on the nature of the failure. For example, if the server cannot resolve the name, it returns a message in the following format:

C:\nslookup <Destination_host>

Server: <fully_qualified_domain_name>

Address: <server_IP_address>

*** <fully_qualified_domain_name> can’t find <Destination_host>: Non-existent domain

In other cases, the requests to the DNS service time out without a reply and returns a message in the following format:

C:\nslookup Valid_Host

Server: [IP_Address]

Address: w.x.y.z

DNS request timed out.

If the server fails to answer the request, Nslookup returns an error message in the following format:

C:\nslookup

*** Can’t find server name for address <IP_Address>: No response from server

*** Default servers are not available.

This message indicates that the DNS server cannot be reached; it does not indicate why the server is unavailable. The server might be offline, the host computer might not have the DNS service enabled, or there might be a hardware or routing problem.

For more information about DNS troubleshooting, see "Introduction to DNS" in this book.

The name resolution problem might be in your LMHOSTS file, which looks for addresses sequentially from the top down. If more than one address is listed for the same host name, TCP/IP returns the first value it encounters, whether that value is accurate or not.

You can find the LMHOSTS file in \%SystemRoot%\System32\Drivers\Etc. Note that this file does not exist by default; a sample file named LMHOSTS.SAM exists. This file must be renamed to LMHOSTS before it is used.

Note While \%SystemRoot%\System32\Drivers\Etc is the default directory for this file, exactly which LMHOSTS file is consulted depends on the value of the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\databasepath registry entry. The database path tells the local computer where to look for the LMHOSTS file.

To determine the cause of long connect times after adding an entry to LMHOSTS, take a look at the order of the entries in the LMHOSTS file.

Long connect times can occur when a large LMHOSTS file has the specified entry at the end of the file. To speed up resolution of the entry, mark the entry in LMHOSTS as a preloaded entry by following the mapping with the #PRE tag (see "Nbtstat" earlier in this chapter for an example). Then use the nbtstat -R command to update the local name cache immediately.

Alternately, you can place the mapping higher in the LMHOSTS file. As discussed in the appendix "LMHOSTS File" in this book, the LMHOSTS file is parsed sequentially from the top to locate entries without the #PRE keyword. Therefore, you should always place frequently used entries near the top of the file and place the #PRE entries near the bottom.

Make sure your computer’s WINS configuration is correct. In particular, check the address for the WINS server.

In the WINS Configuration dialog box, add the server’s IP address (if none is listed) and check to see whether LMHOSTS lookup is enabled. Also check to see whether NetBIOS over TCP/IP is taken from the DHCP server, enabled, or disabled. If you are using DHCP for this host computer, take the value from the DHCP server. Otherwise, enable NetBIOS over TCP/IP.

Unable to Reach an IP Address

If host name resolution occurs successfully, the problem might lie elsewhere. In this case, the problem might be simply a matter of correcting the IP configuration rather than examining the name resolution process.

TCP/IP troubleshooting generally follows a set pattern. In general, first verify that the problem computer’s TCP/IP configuration is correct, and then verify that a connection and a route exist between the computer and destination host by using Ping, as described in "Testing Network Connection with Ping and PathPing" later in this chapter.

Compile a list of what works and what doesn’t work, and then study the list to help isolate the failure. If link reliability is in question, try a large number of pings of various sizes at different times of the day, and plot the success rate or use the PathPing utility. When all else fails, use a protocol analyzer such as Microsoft Network Monitor.

Check Configuration with IPConfig

When troubleshooting a TCP/IP networking problem, begin by checking the TCP/IP configuration on the computer experiencing the problem. Use the ipconfig command to get the host computer configuration information, including the IP address, subnet mask, and default gateway.

When Ipconfig is used with the /all switch, it produces a detailed configuration report for all interfaces, including any configured remote access adapters. Ipconfig output may be redirected to a file and pasted into other documents. To do so, type ipconfig > <directory\file name>. The output is placed in the directory you specified with the file name you specified.

The output of Ipconfig can be reviewed to find any problems in the computer network configuration. For example, if a computer has been configured with an IP address that is a duplicate of an existing IP address that has already been detected, the subnet mask appears as 0.0.0.0.

The following example illustrates the results of an ipconfig /all command on a computer that is configured to use a DHCP server for automatic TCP/IP configuration, and WINS and DNS servers for name resolution:

Windows NT IP Configuration

Ethernet adapter Local Area Connection:

If no problems appear in the TCP/IP configuration, the next step is to test the ability to connect to other host computers on the TCP/IP network.

Test Network Connection with Ping and PathPing

Ping is a tool that helps to verify IP-level connectivity; PathPing is a tool that detects packet loss over multiple-hop trips. When troubleshooting, the ping command is used to send an ICMP Echo Request to a target host name or IP address. Use Ping whenever you want to verify that a host computer can send IP packets to a destination host. You can also use the Ping tool to isolate network hardware problems and incompatible configurations.

Note If you call ipconfig /all and receive a response, there is no need to ping the loopback address and your own IP address — Ipconfig has already done so in order to generate the report.

It is best to verify that a route exists between the local computer and a network host by first using ping and the IP address of the network host to which you want to connect. The command syntax is:

ping <IP address>

Perform the following steps when using Ping:

Note If your local address is returned as 169.254.y.z, you have been assigned an IP address by the Automatic Private IP Addressing (APIPA) feature of Windows 2000. This means that the local DHCP server is not configured properly or cannot be reached from your computer, and an IP address has been assigned automatically with a subnet mask of 255.255.0.0. Enable or correct the DHCP server, restart the local computer, and see if the networking problem persists.

If your local address is returned as 0.0.0.0, the Microsoft MediaSense software override started because the network adapter detects that it is not connected to a network. To correct this problem, turn off MediaSense by making sure that the network adapter and network cable are connected to a hub. If the connection is solid, reinstall the network adapter’s drivers or a new network adapter.

Ping uses host name resolution to resolve a computer name to an IP address, so if pinging by address succeeds, but fails by name, then the problem lies in host name resolution, not network connectivity. For more information about troubleshooting host name resolution, see "Unable to Reach a Host or NetBIOS Name" earlier in this chapter.

If you cannot use Ping successfully at any point, check the following:

If you can ping both the loopback address and your own IP address, the next step is to clear out the ARP cache and reload it. This can be done by using the Arp utility, first to display the cache entries with arp -a or arp -g. Delete the entries with arp -d <IP address>.

Next, look at the default gateway. The gateway address must be on the same network as the local host; if not, no messages from the host computer can be forwarded to any location outside the local network. Next, check to make sure that the default gateway address is correct as entered. Finally, check to see that the default gateway is a router, not just a host, and that it is enabled to forward IP datagrams.

If the default gateway responds correctly, ping a remote host to ensure that network-to-network communications are operating as expected. If this fails, use Tracert to examine the path to the destination. For IP routers that are Windows NT or Windows 2000 computers, use the route utility or the Routing and Remote Access administrative tool on those computers to examine the IP routing table. For IP routers that are not Windows NT or Windows 2000 computers, use the appropriate utility or facility to examine the IP routing table.

Four error messages are commonly returned by Ping during troubleshooting. They are:

The number of hops required to reach the destination exceeds the TTL set by the sending host to forward the packets. The default TTL value for ICMP Echo Requests sent by Ping is 32. In some cases, this is not enough to travel the required number of links to a destination. You can increase the TTL using the -i switch, up to a maximum of 255 links.

If increasing the TTL value fails to resolve the problem, the packets are being forwarded in a routing loop, a circular path among routers. Use Tracert to track down the source of the routing loop, which appears as a repeated series of the same IP addresses in the Tracert report. Next, make an appropriate change to the routing tables, or inform the administrator of a remote router of the problem.

This message indicates one of two problems: either the local system has no route to the desired destination, or a remote router reports that it has no route to the destination. The two problems can be distinguished by the form of the message. If the message is simply "Destination Host Unreachable," then there is no route from the local system, and the packets to be sent were never put on the wire. Use the Route utility to check the local routing table.

If the message is "Reply From <IP address>: Destination Host Unreachable," then the routing problem occurred at a remote router, whose address is indicated by the "<IP address>" field. Use the appropriate utility or facility to check the IP routing table of the router assigned the IP address of <IP address>.

If you pinged using an IP address, retry it with a host name to ensure that the IP address you tried is correct.

This message indicates that no Echo Reply messages were received within the default time of 1 second. This can be due to many different causes; the most common include network congestion, failure of the ARP request, packet filtering, routing error, or a silent discard. Most often, it means that a route back to the sending host has failed. This might be because the destination host does not know the route back to the sending host, or one of the intermediary routers does not know the route back, or even that the destination host’s default gateway does not know the route back. Check the routing table of the destination host to see whether it has a route to the sending host before checking tables at the routers.

If the remote routing tables are correct and contain a valid route back to the sending host, to see if the ARP cache lacks the proper address, use the arp -a command to print the contents of the ARP cache. Also, check the subnet mask to be sure that a remote address has not been interpreted as local.

Next, use Tracert to follow the route to the destination. While Tracert does not record the address of the last hop or the path that the packet followed on the return path, it might show that the packet made it to the destination. If this is the case, the problem is probably a routing issue on the return path. If the trace doesn’t quite reach the destination, it might be because the target host is protected by a firewall. When a firewall protects the destination, ICMP packet filtering prevents the ping packets — or any other ICMP messages — from crossing the firewall and reaching their destination.

To check for network congestion, simply increase the allowed latency by setting a higher wait time with the -w switch, such as 5000 milliseconds. Try to ping the destination again. If the request still times out, congestion is not the problem; an address resolution problem or routing error is a more likely issue.

This error message indicates that the requested host name cannot be resolved to its IP address; check that the name is entered correctly and that the DNS servers can resolve it.

Test IP-to-MAC Address Resolution with ARP

Windows 2000 TCP/IP allows an application to communicate over a network with another computer by using either an IP address, a host name, or a NetBIOS name. However, regardless of which naming convention is used, the destination must ultimately be resolved to a hardware address (media access control (MAC) address) for shared access media such as Ethernet and Token Ring.

The Address Resolution Protocol (ARP) allows a host to find the MAC address of a node with an IP address on the same physical network, when given the node’s IP address. To make ARP efficient, each computer caches IP-to-MAC address mappings to eliminate repetitive ARP broadcast requests.

The Arp tool allows a user to view and modify ARP table entries on the local computer. The arp command is useful for viewing the ARP cache and resolving address resolution problems.

A static entry can be added to an ARP file by issuing the arp -s <IP address> <MAC address> command. However, adding such static ARP cache entries must be used with caution as it is easy to enter the wrong MAC address for an IP address.

When starting up, Windows performs a gratuitous ARP to detect any duplication with its own IP address. While this detects most cases of duplicate IP addresses, in a few situations two TCP/IP hosts (either Microsoft or non-Microsoft) on the same network can be configured for the same IP address.

The MAC and IP address mapping is done by the ARP module, which uses the first ARP response it receives. Therefore, the impostor computer’s reply sometimes comes back before the intended computer’s reply.

These problems are difficult to isolate and track down. Use the arp -a command to display the mappings in the ARP cache. If you know the Ethernet address for the remote computer you wish to use, you can easily determine whether the two match. If not, use the arp -d command to delete the entry, then use Ping with the same address (forcing an ARP), and check the Ethernet address in the cache again by using arp -a.

Troubleshooting the ARP cache can be one of the more difficult tasks in network administration because the problems associated with it are so often intermittent.

The exception to this rule is when you find that the wrong host responds to a command, perhaps a Netuse or Telnet command. The symptoms of invalid entries in the ARP cache are harder to reproduce and involve intermittent problems that only affect a few hosts. The underlying problem is that two computers are using the same IP address on the network. You only see the problems intermittently because the most recent ARP table entry is always the one from the host that responded more quickly to any particular ARP request.

To address the problem, display the ARP table using the arp -a command. Following is an example output of the arp -a command.

C:\>arp -a 172.16.0.142

Interface: 172.16.0.142

Internet address Physical Address Type

172.16.0.1 00-e0-34-c0-a1-40 dynamic

172.16.1.231 00-00-f8-03-6d-65 dynamic

172.16.3.34 08-00-09-dc-82-4a dynamic

172.16.4.53 00-c0-4f-79-49-2b dynamic

157.59.5.102 00-00-f8-03-6c-30 dynamic

Since addresses assigned by DHCP do not cause address conflicts like those described here, the main source of these conflicts is likely to be static IP addresses. Maintaining a list of static addresses (and corresponding MAC addresses) as they are assigned can help you track down any address conflict just by examining the IP and MAC address pairs from the ARP table and comparing them to the recorded values.

If you do not have a record of all IP and MAC address pairs on your network, you might want to examine the manufacturer bytes of the MAC addresses for inconsistencies. These three-byte numbers are called Organizationally Unique Identifiers (OUIs) and are assigned by the Institute of Electrical and Electronics Engineers (IEEE); the first three bytes of each MAC address identify the card’s manufacturer. Knowing what equipment you installed and comparing that with the values returned by arp -a might allow you to determine which static address was entered in error.

Finally, if neither an address pair record nor the manufacturer prefixes reveals the source of the problem, check the Event Viewer for additional clues to the problem. For instance, DHCP might have detected a duplicate card already on the network, and thus denied a computer’s request to join. Other DHCP and related messages here can often quickly isolate and solve a problem.

Verify Persistent Routing Table Entries

The next area to examine are the persistent entries in your routing tables. You can view these using the Route utility. Persistent entries are added using the route add -p command. Use Arp to confirm that the IP address-to-hardware address mappings are correct. If you find an error, change the incorrect entry using the route change command. If you want to make the change permanent, use route add -p. For more information about the Route tool, see "Examine the Routing Table with Route" later in this chapter.

If the local routing table is correct, the problem might be at some point between the host computer and the destination computer. Use Tracert to search for the source of the problem at the router level.

Use Tracert and PathPing

If the routing table configuration is correct, the problem might be with a router or link at any point along the route. You can trace the path to the destination computer using Tracert and PathPing to pinpoint the problem. For more information about using the Tracert tool to examine routing paths, see "Examine Paths with Tracert" later in this chapter.

Use Tracert when you have no connectivity to a site under investigation, since it tells you where connectivity stops. PathPing is more useful when you have connectivity to a site but are experiencing some packet loss or high delay. In these cases, PathPing tells you exactly where packet loss is occurring.

Verify Server Services on the Remote Computer

Sometimes a system configured as a remote gateway or router is not functioning as a router. To confirm that the remote computer you wish to contact is set up to forward packets, you can either examine it with a remote administration tool (assuming that it is a computer you administer) or you can attempt to contact the person who maintains the computer.

You can contact the administrator responsible for a remote network using the databases maintained by InterNIC. The easiest way to do this is to use the Whois tool to find the appropriate person’s name and contact information from the InterNIC database.

Check IP Security on the Initiating Host

IPSec can increase the defenses of a network, but it can also make changing network configurations or troubleshooting problems more difficult. In some cases, IPSec running on the initiating host of a computer under investigation can create difficulties in connecting to a remote host. To determine if this is a source of problems, turn off IPSec and attempt to run the requested network service or function.

If the problem disappears when IPSec policies are turned off, you know that the additional IPSec processing burden or its packet filtering are responsible for the problem. To solve the problem, use the following procedure:

If you need to disable IP Security only for a specific computer, you can disable the IPSec Policy Agent Service on that computer.

For more information about IPSec issues, see "Internet Protocol Security" in this book.

Check Packet Filtering

Any mistakes in packet filtering at the stack, router, proxy server, Routing and Remote Access service, or IPSec level can make address resolution or connectivity fail. To determine if packet filtering is the source of a network problem, you must disable the TCP/IP packet filtering.

Try pinging an address using its DNS name, its NetBIOS name, or its IP address. If the attempt succeeds, the packet filtering options might be misconfigured or might be too restrictive. For instance, the filtering might permit the computer to act as a web server, but might in the process disable tools like Ping or remote administration. Restore a wider range of permissible filtering options by changing the permitted TCP, UDP, and IP port values.

Troubleshooting IP Routing

Windows 2000 supports routing on both single- and multi-homed computers with or without the Routing and Remote Access service. The Routing and Remote Access service includes the Routing Information Protocol (RIP) and the Open Shortest Path First (OSPF) routing protocols. Routers can use RIP or OSPF to dynamically exchange routing information.

This section provides information about the Windows 2000–based routing table as used on single- and multi-homed computers with or without the Routing and Remote Access service. This background information helps with TCP/IP troubleshooting. For more information about TCP/IP routing, see "Unicast IP Routing" in the Windows 2000 Internetworking Guide. For information about troubleshooting IP multicast routing, see "Multicast IP Support" in the Windows 2000 Internetworking Guide.

Cannot Connect to a Specific Server

To determine the cause of connection problems when trying to connect to a specific server using NetBIOS-based connections, use the nbtstat -n command to determine what name the server used to register on the network.

Nbtstat -n output lists several names that the computer has registered. A name resembling the computer’s name as shown on the desktop should be present. If not, try one of the other unique names displayed by Nbtstat.

The Nbtstat tool can also display the cached entries for remote computers from either #PRE entries in the LMHOSTS file or from recently resolved names. If the name the remote computers are using for the server is the same, and the other computers are on a remote subnet, be sure that they have the computer’s mapping in their LMHOSTS files or WINS servers.

Connection to Remote Host Hangs

To determine why a TCP/IP connection to a remote computer is not working properly, use the netstat -a command to show the status of all activity on TCP and UDP ports on the local computer.

A good TCP connection usually shows 0 bytes in the Sent and Received queues. If data is blocked in either queue or if the state is irregular, the connection is probably faulty. If not, you are probably experiencing network or application delay.

Examining the Routing Table with Route

In order for two hosts to exchange IP datagrams, they must both have a route to each other, or use default gateways that know of a route. Normally, routers exchange information with each other using a routing protocol such as RIP.

By default, IP routing is disabled. To enable IP routing, you must allow the computer to forward IP packets it receives. This requires a change to the Windows 2000 system registry. When you enable the Routing and Remote Access service for IP routing, this registry entry is made automatically.

If the Windows 2000 router does not have an interface on a given subnet, it needs a route to get to that subnet. This can be handled using a default route or by adding static routes. For more information about dynamic routing environments, see "Unicast IP Routing" in the Windows 2000 Internetworking Guide.

To add a static route, use the Route utility as follows:

route add 172.16.41.0 mask 255.255.255.0 172.16.40.1 metric 2

In this example, the route add command states that to reach the 172.16.41.0 subnet with a mask of 255.255.255.0, use gateway 172.16.40.1. It also shows that the subnet is two hops away. You may need to add static routes on downstream routers telling packets there how to get back to the 172.16.40.0/24 subnet.

Examine Paths with Tracert

Tracert is a route tracing utility that uses incrementally higher values in the TTL field in the IP header to determine the route from one host to another through a network. It does this by sending ICMP echo request messages and analyzing ICMP error messages that return. Tracert allows you to track the path of a forwarded packet from router to router for up to 30 hops. If a router has failed or if the packet is routed into a loop, Tracert reveals the problem. Once the problem router is found, its administrator can be contacted if it is an offsite router, or the router can be restored to fully functional status if it is under your control.

Troubleshooting Gateways

If you see the message "Your default gateway does not belong to one of the configured interfaces..." during setup, find out whether the default gateway is located on the same logical network as the computer’s network adapter. The easiest way to do this is to compare the network ID portion of the default gateway’s IP address with the network IDs of the computer’s network adapters. In other words, check that the bitwise logical AND of the IP address and the subnet mask equals the bitwise logical AND of the default gateway and the subnet mask.

For example, a computer with a single network adapter configured with an IP address of 172.16.27.139 and a subnet mask of 255.255.0.0 requires a default gateway of the form 172.16.y.z. The network ID of the IP interface is 172.16.0.0/16. Using the subnet mask, TCP/IP can determine that all traffic on this network is local; everything else must be sent to the gateway.

Troubleshooting ARP

Network traffic sometimes fails because a router's proxy ARP request returns the wrong address. A router makes this ARP request on behalf of an IP address on its intenal subnets (just as a remote access server makes a request on the LAN for its remote access clients). The problem is that the router’s proxy ARP requests return the wrong MAC address to the sending host. As a result, the sending host sends its traffic to the wrong MAC address. In other words, the problem stems from proxy ARP replies.

To address this problem, use Network Monitor to capture a trace. If the trace reveals that when a sending host sends an ARP request for the MAC address of a destination IP address, a device (usually a router) replies with a MAC address other than the destination’s correct MAC address.

To determine if this is the problem, check the ARP cache of the source host to make sure it is getting the correct IP address to MAC address resolution. Alternatively, you can capture all traffic with Network Monitor and later filter the captured traffic to display only the ARP and RARP protocols. The RARP protocol converts MAC addresses to IP addresses and is defined in RFC 903.

You can fix the ARP problem by disabling ‘Proxy ARP’ on the offending device. Exactly how this is done depends on the device’s make and model; consult the manufacturer’s documentation.

Troubleshooting Translational Bridging

Allowing nodes from Ethernet segments to communicate with nodes on Token rings presents a number of challenges. Following are the most common problems associated with translational bridging.

The primary factor responsible for problems in this situation are differing MTUs between segments. Token Ring MTUs range from 4,464 to 17,914 bytes, while the Ethernet MTU is 1,500. A FDDI segment has an MTU of 4,532 bytes. When a bridge or Layer 2 switch connects two of these differing networking technologies, packets can be dropped because the Layer 2 switch cannot fragment the data and cannot alert the sending node of the reduced MTU.

In the example shown in Figure 3.6, an Ethernet backbone connects two 16-MB token rings. Instead of a router, a translational bridge in the form of a Layer 2 switch connects the segments. In this case, local traffic on the Token Rings uses an MTU of 17,914 and is not affected by the bridge. However, when Computer A must communicate with Computer B, the bridge drops large packets without notifying Computer A of the need to fragment. In this situation, Computer A has no way to discover the MTU on the other side of the bridge.

In Windows 2000, the MTU registry entry can be adjusted to meet the MTU requirement of the Ethernet segment connecting the two Token Ring segments, reducing all MTUs to the lowest common denominator. Each node’s MTU is reduced to 1,500 to meet the requirements of the Ethernet backbone. However, this solution requires that all traffic (even traffic that is local on a Token Ring) is sent within the reduced MTU.

You can use the Windows 2000 ping -l command to send packets with a defined ICMP Echo Request data size. By sending packets of varying sizes, you can determine the MTU for any given bridge by noting which packet sizes cross the bridge successfully. For example, in Figure 3.6, a ping packet can be sent from Computer A to Computer C with a size of 1,472 bytes, which generates an Echo Reply packet from Computer C. However, if a size of 1,473 bytes or greater is used, the intermediate switch drops the packet. Computer C does not receive the Echo Request and no Echo Reply is generated.

The default ICMP Echo Request contains 32 bytes of data; you can use the ping <IP address or Host Name> -l <data size> command to specify a different data size. For example, you can ping with the maximum Ethernet data size by entering this command:

ping 134.56.78.1. -l 1472

The data size specified by the -l switch is 1,472 rather than the Ethernet IP MTU of 1,500 because 20 bytes are reserved to make room for the IP header and 8 bytes must be allocated for the ICMP Echo Request header.

When you have determined the MTU, you can set the packet size on either side of the bridge by changing the value in the registry entry. The MTU registry entry can be found at:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\Adapter_GUID

For more information about MTU, see "Windows 2000 TCP/IP" in this book.

Troubleshooting PMTU Black Hole Routers

Some routers do not send an "ICMP Destination Unreachable" message when they cannot forward an IP datagram. Instead, they ignore the datagram. Typically, an IP datagram cannot be forwarded because its maximum segment size is too large for the receiving server, and the Don’t Fragment bit is set in the header of the datagram. Routers that ignore these datagrams and send no message are called PMTU black hole routers.

To respond effectively to black hole routers, you must enable the Path MTUBH Detect feature of TCP/IP. Path MTUBH Detect recognizes repeated unacknowledged transmissions and responds by turning off the Don’t Fragment bit. After a datagram is transmitted successfully, it reduces the maximum segment size and turns the Don’t Fragment bit on again.

The Path MTUBH Detect feature is disabled by default, but you can enable it by adding the EnablePMTUBHDetect entry to the registry and setting its value to 1. EnablePMTUBHDetect is an optional entry that does not appear in the registry unless you add it. You must place it in:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip \Parameters.

You can disable Path MTUBH Detect by deleting EnablePMTUBHDetect from the registry or by setting the entry’s value to 0.

A second registry entry, EnablePMTUDiscovery, also helps address the PMTU black hole router problem. This key is enabled by default. EnablePMTUDiscovery completely enables or disables the PMTU discovery mechanism.When PMTU discovery is disabled, a TCP Maximum Segment Size (MSS)of 536 bytes is used for all non-local destination addresses.

The PMTU between two hosts can be discovered manually using the ping -f command, as follows:

ping -f -n <number of pings> -l <size> <destination IP address>

The following example shows how Ping’s size parameter can be varied until the MTU is found. Note that Ping’s size parameter specifies just the size of the ICMP Echo Request data to send, not including the IP and ICMP Echo Request headers. The ICMP Echo Request header is 8 bytes, and the IP header is normally 20 bytes. In the Ethernet case shown here, the link layer MTU contains the maximum-sized Ping buffer plus 28, for a total of 1500 bytes on the first ping and 1501 on the second:

C:\>ping -f -n 1 -l 1472 10.99.99.10

Pinging 10.99.99.10 with 1472 bytes of data:

Reply from 10.99.99.10: bytes=1472 time<10ms TTL=128

Ping statistics for 10.99.99.10:

Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\>ping -f -n 1 -l 1473 10.99.99.10

Pinging 10.99.99.10 with 1473 bytes of data:

Packet needs to be fragmented but DF set.

Ping statistics for 10.99.99.10:

Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),

Approximate round trip times in milli-seconds:

Minimum = 0ms, Maximum = 0ms, Average = 0ms

In the second ping, the IP layer returns an ICMP error message that Ping interprets. If the router had been a black hole router, Ping would not be answered once its size exceeded the MTU that the router could handle. Ping can be used in this manner to detect such a router.

Troubleshooting Services

In addition to its role in providing basic network communications, TCP/IP is the cornerstone of a number of network services such as Routing and remote Access, printing, IP Security, and the Browser Service. These services are discussed in more detail in other chapters, but a few examples of basic troubleshooting for these services are described below.

Cannot Ping Across a Router as a Remote Access Client

This problem occurs if you have selected Use default gateway on remote network under the General tab of the Advanced Internet Protocol (TCP/IP) Properties in the Dial-Up Connections page. This feature adds a default route to the routing table with a metric of 1 and changes the existing default route to a metric of 2. All non-local traffic is now forwarded to the gateway on the remote access link. However, to access the Internet, this feature must be enabled.

To ping or otherwise connect to computers in a remote subnet across a router while you are connected as a remote access client to a remote Windows remote access server, use the route add command to add the route of the subnet you want to use.

Troubleshooting TCP/IP Database Files

Table 3.13 lists the UNIX-style database files that are stored in the %SystemRoot%\System32\Drivers\Etc directory when you install Microsoft TCP/IP:

To troubleshoot any of these files on a local computer, make sure the entry format in each file matches the format defined in the sample file originally installed with Microsoft TCP/IP. Check for spelling errors, invalid IP addresses, and identifiers.

Removing and Reinstalling TCP/IP

When you attempt to reinstall a TCP/IP service, you might receive a "The registry subkey already exists" error message. To correct this problem, you should ensure that all the components of a given TCP/IP service are properly removed and then remove the appropriate registry subkeys.

Caution Do not use a registry editor to edit the registry directly unless you have no alternative. The registry editors bypass the standard safeguards provided by administrative tools. These safeguards prevent you from entering conflicting settings or settings that are likely to degrade performance or damage your system. Editing the registry directly can have serious, unexpected consequences that can prevent the system from starting and require that you reinstall Windows 2000. To configure or customize Windows 2000, use the programs in Control Panel or Microsoft Management Console (MMC) whenever possible.

If you removed TCP/IP and its related service components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NetBT

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TcpipCU

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Dhcp

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LmHosts

If you removed the SNMP service components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RFC1156Agent

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Snmp

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Snmp

If you removed the TCP/IP Printing service components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Lpdsvc

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\TcpPrint

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ LpdsvcSimple TCP/IP Services

If you removed the Simple TCP/IP Services components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SimpTcp

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SimpTcp

If you removed the DHCP service components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DhcpMibAgent

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DhcpServer

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DhcpServer

If you removed the WINS service components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wins

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WinsMibAgent

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Wins

If you have removed the DNS service components, you must also remove the following registry subkeys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Dns

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DnsMibAgent

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Dns