The Road Runner service requires the user computer to "log in" to the service before it can send any packets to the outside network. This is apparently implemented in the TAS with a packet filtering firewall in the upstream direction. Before login, one can send ICMP packets (e.g., ping) to the outside Internet, but nearly all other upstream TCP and UDP packets are blocked. The user computer can only speak to the local DNS/login server. Downstream packets do not appear to be filtered or blocked at any time.
The official login program exists only for Windows and Mac-OS; there is no support for Linux, BSD or any other operating system.
The login program, and the protocol it implements, are truly wretched abortions that should never have seen the light of day. Chronic problems with this braindamage comprise the majority of complaints to the local roadrunner.help newsgroup. And every time RR tries to "fix" the protocol, things only seem to get worse.
For example, the login program sends regular "heartbeat" UDP packets to the TAS after logging in. If this is not done, the TAS silently logs you out. Heartbeats are never acknowledged, so there is no way to tell if, say, packet losses on the reverse channel have logged you off except by noticing that all your network accesses time out -- even though the login program is still running. You must then kill and restart the login program.
The heartbeat packets also wreak havoc on the Motorola cable modem's adaptive reverse channel polling algorithms. Because many users remain logged in but idle for long periods, heartbeats produce a significant and wholly unnecessary load on the reverse links.
To alleviate these two problems, RR released a new version of the login program that lengthened the heartbeat interval from an original value of 30 sec to about 5 minutes, and they lengthened the timeout interval in the TAS to about 20 minutes. But this aggravated another problem! If you try to log in when the TAS thinks you're already logged in, the TAS rejects the request and the login program aborts. You have to wait until you "time out" of the TAS, but thanks to the increased timeout interval, this now takes as long as 20 minutes! That's because the login protocol lacks a feature basic to any "real" protocol like TCP, i.e., a "reset" feature designed to deal with "half open" associations.
Many RR users, pointing out that the @Home service has no login program, have repeatedly asked Time Warner to get rid of the login requirement. Yet Road Runner management (particularly Mario Vecchi, Excalibur head honcho), has turned a deaf ear, giving no rational explanation whatsoever for retaining it. No wonder Scott Adams paints such a vivid image in The Dilbert Future of cable company executives cleaning their guns and painting targets on their shoes.
Update: At the end of July, Road Runner modified the login protocol and program once again to essentially do away with periodic heartbeats. It is not clear if heartbeats are actually never necessary or are still required at very infrequent intervals (every 32760 minutes).
Workaround: Use my Road Runner login program, available in C code that compiles and runs under Linux, BSDI and FreeBSD. This program is a substantial improvement over the official login program. It periodically checks to see if you're still logged in (by attempting a TCP connection to a host that is unreachable when you're logged out) and automatically logs in again if necessary.
My program also ignores the responses from the TAS during the login process. So if you run it when you're already logged in, it will simply "fall through" to the sending of the heartbeats to keep you logged in.
The current version of rrlogin.c has all of the San Diego Road Runner IP addresses built into it. Once you have started the DHCP client, obtained an IP address from Road Runner and installed the appropriate routing entries, the rrlogin program will automatically determine the correct IP address to use for the login protocol.
Workaround: Use an alternate path for your upstream path, such as a dialup connection to an Internet service provider. The pre-login packet filter only blocks upstream packets, so if you can find another way to get your upstream packets to the Internet you will still be able to receive downstream packets via Road Runner.
Drawback: The cost of maintaining an ISP account. The cost of a modem and extra phone line. The slower speed of a modem (28.8kb/s) as compared to the Road Runner reverse channel (768kb/s).
My login program now works well enough to make the login requirement a petty nuisance rather than a serious problem, at least for systems that can compile and run a C program (e.g., standalone routers are still a problem).
On Friday June 12, 1998. Road Runner San Diego posted to our Network Status page and to the roadrunner.help newsgroup that port 25 would be filtered in our system beginning July 12, 1998.The following is the original description of the problem:This filtering would have affected a small percentage of customers who run private email servers in their homes along with customers which access certain chat areas. The customers affected would have been required to upgrade their service to a commercial account and sign a separate contract with Road Runner.
This proposed change to the system was brought about by the increasing amount of unsolicited bulk email (SPAM) being relayed through unprotected private customer servers in our system.
Each Road Runner division across the country sets its own policy, and many divisions do restrict port 25 activity. Road Runner San Diego does listen to and values its customers input. Due in part to customer feedback, we have elected to indefinitely postpone the filtering of port 25.
Road Runner San Diego will continue to take necessary steps to protect its network and customers against any form of system abuse and denial of service activities both from external and internal sources. We will take increasingly aggressive action to identify and isolate individuals who do not comply with our acceptable use policy in its broadest sense. We are committed to swift response concerning subscriber conduct issues.
We are in the process of completing an Acceptable Use Policy which will be posted when approved.
We continue to strive to provide you with the best on-line experience possible by making sure our equipment is scaled properly, redundant, and powerful enough to handle our rapid growth.
Edwin Kruse
Network Services Supervisor
RR San Diego
On Thursday, June 11, 1998, Road Runner summarily blocked downstream traffic to TCP port 25 (SMTP). They took this action without the courtesy of an advance warning.
On June 12, the following notice appeared on the Road Runner status page:
6/12/98 3:40pm Road Runner has stopped filtering Port 25 on our system as of 3:00pm on 6/12/98. This filtering took place for approximately 24 hours. Filtering port 25 prevented customers from receiving email on email servers they had set up in their homes. Some commercial accounts were also affected.While I understand that spam can be a problem, I find it interesting that they'll waive the filter only if you "upgrade" to a much more expensive "commercial" account -- even if you run a personal SMTP server that is already configured to prevent relaying of spam.Extremely aggressive behavior by persons outside of our system using customer's unprotected email servers to relay spam (Unsolicited Bulk Email), brought about this response. In order to protect our system and our customers, we found it necessary to implement this filter temporarily without notice.
Unsolicited bulk email relaying is an ongoing problem. In thirty days (07/12/98), filtering of Port 25 will resume. Only commercial accounts with long-lease IPs will be excepted from filtering. Residential customers wishing to utilize SMTP servers from their home will need to upgrade to a commercial account. For more information please contact us at 695-3220 and ask for the Business Solutions desk.
We regret any inconvenience the filtering may have caused.
Edwin Kruse
Network Services Supervisor
RR San Diego
Workaround: You could use the RR SMTP and POP servers instead of running your own SMTP service. They've been notably slow and unreliable in the past, but recent reports say they've improved. I wouldn't know, because I rarely use them; in any event, they cannot possibly perform better than direct delivery.
Workaround: Use IP-in-IP tunneling. This is the only really effective workaround I know of at present. But this requires a remote tunnel system on a "friendly" network and can take a fair bit of know-how and work to set up. For the details, see my paper IP-in-IP Tunneling Over Road Runner With Linux. This approach does have the advantage of giving you a fixed IP address on which to receive your mail, which is better than (re)registering your temporary RR IP address with a dynamic DNS service.
(On March 30, 1998, the CERFnet link bandwidth was increased from 10 Mb/s to 43 Mb/s (DS3).
Unlike most ISPs, the RR system does not provide a web server for the use of its subscribers in hosting their own personal web pages. This has resulted in many users setting up their own web servers, which stresses the limited upstream capacity of the Motorola cable modems.
Workaround: Find another site outside RR to host your web pages.
I strongly recommend against running public-access web servers on the cable modem service. The total upstream capacity for any given cable modem channel is only 768 kb/s. While this is plenty for HTTP requests, interactive sessions, outbound mail and the occasional manual file transfer, it could easily be swamped by the sustained outbound traffic of a popular personal web server.
I understand how tempting it is to run your own web server given the self-defeating reluctance of RR management to provide one at the head end for its customers, but please keep in mind the effect your personal web server can have on your fellow RR subscribers. Until RR management gets its act together, we have to share what we have as efficiently as possible.
Road Runner is promising personal web pages by the end of July. We'll see.
The San Diego Road Runner system is connected to the external Internet through MCI. Many important Internet sites in San Diego connect instead to CERFnet, including Qualcomm, SAIC, UCSD and SDSC -- all of whom have many employees who are undoubtedly interested in using Road Runner to work from home. But the closest interconnection between CERFnet and MCI is at MAE-West in San Jose. This means inefficient connectivity between Road Runner and many San Diego sites.
On April 16, 1997, Road Runner installed extra bandwidth (3 T1s vs 1 T1) on their connection to MCI. On or around the same time, the packet loss rate into MCI at MAE-West, which had been an unusable 30% for months, dropped to 0-2%, even in the afternoon. Connectivity to CERFnet is now dramatically better, and the improvement seems to be holding. So I'm downgrading this problem from 3 "frowny faces" to 2, and rewriting this entry to reflect the current status.
On July 18, 1997, after several months of increasing load and steadily worsening performance, Road Runner installed 4 additional T1 links to MCI, making 7 total. Connectivity to the external Internet has noticeably improved.
Road Runner has announced that they will be installing a 4 Mb/s link to CERFnet by the end of July 1997. It remains to be seen if this actually happens as promised.
The San Diego Road Runner system is connected to the external Internet through MCI. Many other local Internet sites connect instead to CERFnet; this includes Qualcomm, UCSD and SDSC. The closest interconnection between CERFnet and MCI is at MAE-West in San Jose, which is notoriously overloaded; packet loss rates consistently run at 30% during the day. This makes the service next to useless for telecommuting, or for any other activity requiring good connectivity to local Internet sites.
It as if Road Runner built an 8-lane driveway to your door, but refused to connect it with any of the existing local streets or freeways. Instead it leads -- without any exits but passing plenty of Time-Warner billboards -- to a garbage dump in San Jose, where you are dumped onto the local streets and expected to make it back to San Diego on your own.
The only solution to this problem is to improve the connectivity between Road Runner and other local San Diego ISPs. We have brought this up repeatedly and urgently with Road Runner management, emphasizing that it makes their service totally unusable for a large segment of their potential market. Yet they seem strangely unconcerned and unwilling to solve the problem, even though we know that fiber and router facilities already exist for a local connection.
Workaround: A stopgap solution for those who already have reasonable connectivity to work (e.g., ISDN) is to multi-home their hosts such that traffic to and from work is routed over ISDN while some or all outside Internet traffic is routed to Road Runner. If there are certain Internet sites that you access frequently, test both paths. In some cases Road Runner may outperform your work connection, especially if the site is on MCInet not far from San Diego (placing it at the far end of the MAE-West black hole as seen from CERFnet).
Update: On March 18, 1997 I noticed the path between CERFnet and MCI through MAE-West was substantially better. Packet loss rates at 3pm (normally the heaviest time of the day) were only 1%. This is a substantial improvement, but the cause is as yet unknown. There is still the need for a direct connection between Road Runner and the major San Diego area Internet sites, if for no other reason than to avoid the inefficient use of the links to San Jose.
Update: On March 31, 1997, the CERFnet->MCI connection seems to have relapsed. It is again losing ~30% of all packets at most times of the day. File transfers proceed extremely slowly, and abort frequently.
Ping experiments typically show one of three distinct behaviors:
--- 204.210.0.1 ping statistics --- 388 packets transmitted, 388 packets received, 0% packet loss round-trip min/avg/max = 5.8/127.9/1138.1 ms
This discrete-time behavior did not seem affected by either the size of the ping packets or their spacing. This strongly suggests that the cable modems were being polled once every 2 seconds and any packets queued for transmission were picked up at that time. I am assuming without actual proof that only reverse (upstream) link delay was significant, for two reasons: multiple-access contention is not an issue on the forward (downstream) link, and it is also much faster than the reverse link -- 27 megabits/s vs 768 kilobits/s. I would like to verify this by using an alternate (ISDN) upstream path with the RR downstream path, but the poor RR connectivity to CERFnet again presents a serious obstacle.
The "ramping" behavior can be explained by the presence of a second, faster polling rate of about 260 ms. The "ramp" is simply an artifact of the polling rate not being an exact multiple of the ping rate (1 ping/sec).
The polling rate appears to depend on user activity. The first "ping" after the modem has been idle for some time is consistently slower than subsequent pings. This seems like a reasonable way to build a multiple access system with many terminals, most of which are idle at any given time. To minimize the time spent polling idle terminals, those terminals are polled relatively infrequently until they start producing traffic; once the terminal has traffic, it is polled more frequently until it no longer does. This takes advantage of the bursty nature of computer network traffic; if you send a packet at time T, the probability that you'll soon send another packet is much greater than if you hadn't sent that packet at time T.
This behavior does, however, lessen one of RR's claimed advantages over other methods of Internet access. Indeed, the idle polling delay seems to be a significant fraction of the delay in bringing up a demand-dialed ISDN circuit.
Internet Service Providers generally allow access to their POP (Post Office Protocol) servers from outside their system so their subscribers can access their mail from other parts of the Internet, e.g., when traveling. Not Road Runner -- the POP port is firewalled from the outside Internet. It can only be accessed from a Road Runner cable modem.
Workaround: Don't use the Road Runner mail service; receive your mail at work or through a different ISP, and access that service's POP server via Road Runner when you're home.
Drawback: Extra cost if you have to maintain a separate ISP account just for email. Poor performance through Road Runner if the other ISP is not easily reached from MCInet (Road Runner's provider).
Workaround: Run a UNIX (Linux, BSD) system at home that can be accessed remotely from the external Internet. Option 1: Log into that system interactively to read your mail with a user agent that can be used from a remote X or TELNET terminal, such as Emacs, Pine, MH, etc. Option 2: Set up a TCP forwarding connection with SSH so you can run a local email agent (e.g., Eudora, Netscape Mail) while making the connection to the POP server appear as if it's coming from your home system. Here is a sample slogin command that could be used on a roving laptop to provide proxy access to the Road Runner POP server through a home system:
slogin -C -L 110:mail.san.rr.com:110 204.210.35.35This assumes that the IP address assigned by Road Runner to your home system (which must be running the SSH daemon) is 204.210.35.35. You can substitute the IP address of mail.san.rr.com (currently 204.210.0.1) in place of its domain name if you like, or if the Road Runner DNS servers aren't cooperating. Note also that executing the above command on a UNIX-like client system generally requires that you be root, as UNIX considers port 110 to be "privileged". The Windows 95 SSH client doesn't have this restriction.
You would then configure your local news reader and mail client (e.g., Eudora) to use the loopback address (127.0.0.1) as their servers. SSH will then automatically forward connections to these local ports to your home system, which will in turn forward them to the Road Runner servers.
Using SSH in this manner has the added advantage of automatically compressing and encrypting everything between your roving laptop and your home system, though the path between your home system and the Road Runner servers is still in the clear.
Drawback: You need to know the dynamic IP address of your home system. If it (or the RR system) crashes and reboots, your home machine may get a new IP address -- which you would not necessarily know when traveling.
Drawback: The noise and cost (electricity, etc) of keeping a machine running full time at home; if you live alone and it crashes, you may have no easy way to reboot it remotely. Note: The discussion for POP also applies to NNTP (netnews). Although other news servers may be available when traveling, the history files of some news clients are keyed to the local article numbers of a particular news server, so using an alternate server may not be practical. If so, the discussion for POP applies to NNTP. The slogin command will accept multiple -L options for forwarding more than one TCP port, e.g., both NNTP and POP3.
Update (3/31/97) There now seems to be full intra-system IP connectivity. Traffic to other "subnets" (as defined by the subnet mask handed out by DHCP) goes normally via the default router. The default router also responds to ARP requests for hosts on the same subnet with its own Ethernet address, so this traffic is also correctly handled. If you happen to have another host on the same modem that shares the same subnet, the default router correctly does not respond to ARP requests for this host, thus allowing this traffic to go directly. However, if the two hosts sharing a modem were assigned addresses on different subnets they have to communicate through the RR router, which is inefficient.
The following is the original description of the problem:
Road Runner users seem to be unable to communicate directly with each other at the IP level, even when they can both communicate with an external network. The behavior is the exact reverse of the firewalls on the Road Runner servers; while those servers can only be accessed from within the system, users can only be accessed from outside the system.
Workaround: Set up a tunnel to an external host and encapsulate all intra-RR traffic to it. The far-end tunnel will unwrap the encapsulated packets and feed them back into the RR system from the outside.
The firewall on RR servers, combined with the "reverse firewall" on RR users, means you have to be very careful in setting up your routing tables so that only the RR user traffic, and not the server traffic, is sent into the tunnel.
Drawback: The usual tunneling drawbacks, only worse. If the far end tunnel is on the other side of the MAE-West black hole, then a round trip between two RR users must transit MAE-West twice. Plus, both RR users must implement the tunnel encapsulator in order to communicate directly.
On Friday, March 7 1997, Road Runner management instituted a packet filter that blocks upstream packets addressed to TCP port 80 (HTTP), forcing users to go through their HTTP caching proxy servers.
RR management has been singularly heavy-handed on this issue, completely ignoring numerous complaints from users having trouble getting current information from frequently updated web sites. So this problem seems likely to remain for some time.
Workaround: Use the Road Runner proxy servers (proxy-server.san.rr.com, port 8080). Most of the time they actually work. It's a good idea to use proxy caches whenever possible, especially for popular web sites. They reduce the load on those web sites and on Road Runner's links to the outside Internet.
If you get stale data, click "reload" on your browser until you get a fresh copy.
Drawback: Road Runner management could easily collect logs of your web surfing activity.
Drawback: If the Road Runner proxies go down, you are out of luck.
Workaround: Use another proxy web server that doesn't listen on TCP port 80. One candidate is oceana.sdsc.edu port 3128.
Drawback: The path to SDSC from Road Runner is via MAE-West in San Jose, which is notoriously overloaded. The round-trip packet loss rate over this path is typically 30%. Performance is likely to be very poor.
Workaround: Construct an IP-in-IP "tunnel" that bypasses the Road Runner packet filter and route your upstream HTTP packets through it. Tunneled upstream packets are not blocked by the Road Runner filter, and there appears to be no downstream blocking at all. IP-in-IP tunneling (protocol 4) is implemented in Linux, FreeBSD, NetBSD and KA9Q NOS.
Drawback: No known tunnel implementations exist for Windows 95 and Mac-OS.
Drawback: Requires a cooperating site outside the Road Runner system to host the other end of the tunnel.
Drawback: Most local candidates for a tunnel host, including Qualcomm, UCSD and SDSC, are on CERFnet. This means the path from Road Runner again goes through MAE-West, with all the attendant inefficiences and poor performance.
A packet trace shows that the proxy server tries to open a TCP connection to the personal server, but the reply packets from the personal server don't make it back to the proxy server.
The culprit appears to be overly restrictive upstream packet firewalling in the TAS between the Road Runner user and the proxy server. The only upstream packets allowed to reach the proxy servers are those to TCP destination port 8080, the proxy server port. When the proxy server itself opens a connection to a remote server, it does so from an arbitrary source port. Reply packets from the remote server must be addressed to that particular port. Since this port is almost certainly something other than 8080, reply packets from a personal Road Runner web server are blocked from reaching the proxy server, even though they're simply answering a connection request from the proxy server itself.
I reported this problem in detail to Road Runner months ago. They seem wholly uninterested in fixing it.
Workaround: Disable the proxy server when accessing personal web servers.
Workaround: Specify an external proxy server, e.g., oceana.nlanr.net
port 3128. External sites such as this have no trouble talking to personal
web servers.
The RR system uses the Domain Name System in a rather strange, nonstandard way: the response to a query for "proxy-server.san.rr.com" depends on whether you are logged in or not. In the logged-in state, this query returns a chain of CNAME records eventually giving one of the addresses 204.210.0.21, .22 or .23, presumably to load-balance users among multiple servers. But if the user is not yet logged in, the same query produces a different set of results: 204.210.0.10 or .11.
The apparent intent is to steer the non-logged-in user to a "basic" proxy server, while the logged-in user gets an "enhanced" server. This speculation is reinforced by the canonical names associated with these two groups of addresses: "proxyb" (basic) and "proxye" (enhanced).
The first link in the chain of CNAME records, the one that maps "proxy-server" to "proxye" or "proxyb" has a TTL (Time to Live) of 86400 seconds or 1 day; if a client happens to cache that record it could easily fail to talk to the right proxy server. Presumably this is isn't a problem for the systems for which RR was intended, as Win 95 and MacOS have primitive, non-caching DNS clients. But if you forget to log in before you start your web browser, you may find it trying to use the IP address of the "basic" proxy even after you have logged in.
Workaround: If you have a problem, the simple workaround is to
hard-wire one of the "extended" proxy server IP addresses in your browser.