Road Runner Problem List

Phil Karn
Original version: 10 March 1997
Last Updated: 31 Jan 2000

The name Road Runner for Southwestern Cable's new Internet cable modem service is remarkably apt. While tantalizing its users with (occasional) bursts of blazing speed, its numerous problems, misfeatures and outright brain damage leaves its users feeling much like Wile E. Coyote -- hungry, frustrated, and fanatically determined to overcome every setback. Some of these problems are merely petty annoyances. Past problems have been serious enough to render the service essentially unusable for weeks at a time.

The purpose of this page is to summarize the known problems with Road Runner and to suggest workarounds, if possible, along with their known drawbacks, if any.

To give credit where it is due, I am updating these pages as these problems are fixed. Here is the list of Fixed Road Runner Problems

Key to problem severities:

Minor oddity or nuisance, easily avoided or worked around
Major annoyance, workarounds involve considerable effort, decreased efficiency, and/or loss of functionality
A real show stopper; no workaround -- renders the service nearly useless

These rankings are admittedly somewhat subjective. Because I have access to other servers to provide netnews and to host my web pages, I have not rated these particular drawbacks as severely as others might have. On the other hand, the fact that these problems can be bypassed arguably makes them less severe than, say, cable plant problems or inadequate connectivity to the outside world. 
 

Problem: RR policy prohibits DNS registration of IP addresses

On July 13, the following note appeared on the Road Runner status page:

Note:All IP addresses within the 204.210.xxx.xxx domain are the sole property of Road Runner and Southwestern Cable TV. Customers may not register our IP addresses without prior written consent. If you desire to register a domain name, please call 695-3220 and ask for Road Runner Business Solutions. Road Runner San Diego will act on a case by case basis with regard to violation of this policy.
This is not actually a technical problem per se. It's just another ill-considered and autocratic edict from a laughably inept cable TV company that seems bent on imposing its will on the Internet with only the dimmest understanding of how it all works. And they'll never concede that the "problem" they're trying to solve is the direct result of their own refusal to hand out static IP addresses (as the far more clueful @Home does).

There is of course no way that SWCTV can control entries in other people's DNS databases, any more than they can control who links to their web page -- which is not that different from what they're trying to do here. In fact, it's even harder to discover a DNS record that points to a given IP address than it is to discover a link that points to your web site.

In all seriousness, it is a good idea to avoid registering a RR IP address in the DNS in such a way that you cannot change it quickly when your address changes. Otherwise, people may not be able to reach you (and another user could get your traffic) for some time after your IP address changes. If you do register your IP address in the DNS, do it on a server that you control (i.e., don't give them to the NIC to put into the root servers) and be sure to use very short time-to-live (TTL) fields so that obsolete records won't linger too long.

But just to have some fun with this one, here are some "workarounds" that will give you the same functionality while avoiding the letter of their "prohibition":

Workaround: Don't register your domain name directly in the DNS with an A (address) record and your Road Runner IP address. Instead register a CNAME (canonical name) record that cites the corresponding Road Runner domain name. E.g., instead of the record 

homer   IN      A       204.210.35.42
create the record 
homer   IN      CNAME   dt060n2a.san.rr.com.
Since the rule only mentions IP addresses, not domain names, you're within the strict letter of the law. This is actually a good idea anyway, to avoid confusing some overly paranoid machines that do reverse (PTR) DNS lookups on incoming IP addresses.

Workaround: Create a web page with links to your RR IP address and/or domain name, and change it whenever your address changes. Since this doesn't count as "registration" (at least not in the DNS), again you're not violating the letter of their rule. Again, keep the cache time limits on this page short.

Workaround: Set up an IP-in-IP tunnel with a block of static IP addresses belonging to some other network, and register those in the DNS.

This approach has several advantages. First, the fact that you're using Road Runner for your Internet connection is hidden from those who access your system from the outside, including RR management; it looks like any other system on the "outside" Internet. Second, your addresses remain static even when your RR IP address changes, though you do have to change the routing/encapsulation entry in the remote tunnel system whenever your RR IP address changes. See my page IP-in-IP Tunneling Over Road Runner With Linux for the details. 

Problem: ARP denial-of-service vulnerability

The Motorola Cable Routers at the RR hubs use ARP to resolve user Ethernet addresses. This makes it possible for one user to "hijack" an IP address belonging to another on the same hub and deny him service. This usually happens by accident when someone kills their DHCP client and then continues to use an earlier assigned IP address past the expiration of its lease. If the address is then reassigned to another user, both will issue ARP packets (requests and responses) asserting ownership of the IP address in question. Whoever sends the last assertion is sent all the downstream packets for that IP address until the next assertion is sent.

This vulnerability is inherent in ARP. It is usually not a serious problem because ARP messages are limited to a single LAN under common administrative control, and the users are not mutually hostile. A metropolitan area cable network like RR is clearly somewhat different.

RR could solve this problem completely by integrating the ARP implementation in the cable router with the DHCP server in the TAS. When a user requests an IP address, the DHCP server should "hardwire" the requesting Ethernet address into the cable router's ARP table. It should stay there until the DHCP lease expires; there is no need to issue downstream ARP requests, and it should ignore the assertions made in upstream ARP requests.

Some other cable modem services avoid this problem in a different way: they hard-wire the user's Ethernet address into the cable router at subscription time. (Metro-1 reportedly does this.)

So until the problem can be fixed, all RR users should take special care to ensure that their DHCP clients are running whenever their computers are on line!

Problem: Unreliable Usenet, Mail and web servers

Many users complain that the Road Runner mail (SMTP and POP) servers frequently refuse connections for long periods, and the servers take hours to deliver even short pieces of mail. The problem seems to be inadequate resources combined with heavy spamming. RR management recently blocked SMTP relaying, which seems to have alleviated some of the spam loading, but complaints persist.

The Road Runner USENET server frequently runs out of disk space, usually on the file system with its log files. Although occasional problems of this sort can be expected with any server, the fact that this seems to happen repeatedly and take the better part of a day to fix each time makes it worthy of mention. Other users report that the USENET server does not seem to get a reliable feed of articles, which could certainly be explained by file systems that are too small.

The Road Runner web proxy servers also seem to fail frequently. Many users report that loading pages through at least two of the four servers is frequently extremely slow, taking minutes to load a page that can be loaded in a few seconds through oceana.nlanr.net:3128 or with no proxy at all. The RR sysadmins state that this is due to a known bug (a memory leak) in the cache server software and that it will be fixed "soon", but this problem has existed for at least several months.

Workaround: Use non-RR Mail, USENET and web proxy servers. Or don't use a web proxy server at all now that it is no longer mandatory. Obviously the severity of this problem depends on whether you have access to other servers. Drawback: The whole point of providing such services at the RR head end is to reduce traffic on their link to MCI and on the external Internet. Going to external servers defeats this. 

Problem: High reverse packet loss rates  (where applicable)

Users regularly complain of high packet loss rates. These are usually, but not always, intermittent. Sometimes they are caused by backhaul or Internet congestion, so they affect everyone on the system reaching certain remote sites. RR occasionally adds bandwidth to their Internet connections, but they always seem to do it in a reactive fashion. I.e., they wait until their existing links are totally saturated and everyone complains before they add more capacity.

Sometimes packet losses are due to problems in the cable plant. Typically this afflicts individual users, affecting traffic to all destinations.

Road Runner seems to have little or no proactive network monitoring to detect packet loss problems before the users complain. So it's up to the user to isolate a problem for him/herself. How? With the  traceroute and ping commands.

There is no reason to accept anything less than a 100% success rate when pinging your local cable router. If you encounter losses, call up Southwestern Cable, open up a trouble ticket, and keep on them until they fix the problem. 

Problem: Dynamic IP addressing

The Road Runner service requires that every computer implement the Dynamic Host Configuration Protocol (DHCP) so that it can obtain a temporary IP address assignment from the system. The only way to get a static (permanent) IP address is to buy an overpriced "commercial" account.

Workaround: For client-only systems (e.g., Windows-95 and MacOS) dynamic addresses are generally not a problem.

Workaround: For servers (e.g., UNIX boxes), set up a tunnel interface to a peer on a "friendly" remote network. The tunnel makes it look as though the system has a direct interface on the remote network, with a permanent IP address belonging to that network. This can be done manually with the tunnel drivers provided in Linux and some BSD variants, or it can be done more automatically with the new Mobile IP protocol.

Drawback: The usual drawbacks of tunneling: extra complexity and decreased performance, especially if the tunnel path is through a long or overloaded Internet path -- as is usually the case. 

Problem: Only one computer per cable modem

The Road Runner system is by default restricted to a single computer per cable modem.

Workaround: IP-in-IP tunneling and Network Address Translation (NAT) are both viable solutions. The tunnel/NAT box connects to Road Runner with one Ethernet interface and to the local LAN with the other. In the tunnel mode, it encapsulates IP datagrams from secondary machines on the LAN and tunnels them to a cooperating router on the external Internet. It also accepts encapsulated packets from the remote tunnel, unwraps them and forwards them to the LAN.

In the NAT mode, it replaces the original IP source address and TCP or UDP source port numbers before forwarding each packet to Road Runner, thus making them appear as if they had come from the NAT system itself. The NAT box keeps track of the original addresses and port numbers so incoming reply packets can have their original values restored.

The NAT mode is significantly simpler and more efficient than IP-in-IP tunneling. Non-routable (private) IP addresses can be used on your secondary machines, you don't need a cooperating remote tunnel machine, and traffic is routed optimally. But there's a catch. Machines using a NAT relay must initiate every network transaction. There is no way for an external host on the Internet to initiate a new TCP connection or UDP transaction directly to a secondary machine, even at that machine's request. This is a problem for certain applications (e.g., FTP, Real Audio, ICQ). Some of these applications provide configuration workarounds that avoid the problem, but others may not.

IP-in-IP tunneling is architecturally much cleaner than NAT. It gives your secondary machines a "virtual connection" to some remote network where there is a cooperating tunnel machine. All Internet services (clients and servers) will work unmodified. With tunneling, your secondary machines use IP addresses belonging to the remote network, and it requires that all traffic be routed through the remote tunnel. This is tunneling's major catch. Performance is poor if the path to the tunnel is slow or unreliable, and you must depend on another machine (the remote tunnel).

Once again, Linux has the mechanisms required. See the Linux IPFW(4) manual page for the firewall "masquerade" feature. 

Problem: Inefficient intra-house connectivity

This is a remaining vestige of a now-fixed problem (see No intra-net connectivity on the list of fixed problems.)

Note: This problem is moot when the user is restricted to only one computer per modem. Setting up a separate in-house LAN is the only solution.

This problem is caused by the incorrect subnet mask assigned by DHCP at boot time. Road Runner addresses assigned in a given area of the city come from a block of four Class-C subnetworks. E.g., in the University City area, addresses are assigned in the range 204.210.35.xxx to 204.210.38.xxx. But the netmask is given as 255.255.255.0. If two computers in the same household (i.e., sharing the same cable modem through an Ethernet hub) are assigned IP addresses on the same subnets, all is well; they can communicate directly over the local Ethernet, assuming the client knows the server's temporary IP address. But if DHCP happens to assign them addresses in different subnets, they must communicate via the Road Runner modem, which is far less efficient than communicating directly. Each host thinks the other is not local, so it passes the packet to the RR router.

Workaround: Set up two subnets and dual-home your computers on both such that intra-home traffic goes on a subnet separate from the one connected to the Road Runner modem.

Drawback: Cost, complexity.

Workaround: Set up one computer with two Ethernet interfaces as a Network Address Translator. (This function is included in Linux, which calls it "IP masquerading.") Give this computer the Road Runner connection on one Ethernet interface, and connect the household subnet to the other interface. Configure the NAT function to relay packets as required between Road Runner and computers on the internal network.

Drawback: Architectural inelegance (e.g., breaks certain applications where clients can accept incoming connections). If the NAT machine goes down, you lose your external connectivity from the other machines on your network. 

Back to Phil Karn's Road Runner Page

Back to Phil Karn's Home Page