Simeon mentioned Open Access and in doing so provided a sleight-of-hand reference to Neology in a post of his, after Joe, posted “Broadband in Joe’s world…”

Having read some of Joe’s views on what a network should look like, and stumbling across “no routing, no PPPoE lameness” I felt that I had to re-examine whether PPPoE is lame, and whether PPP itsself, is dead. It also made me thought more in-depth about my vision of a modern open access network. Back when Neology did it’s OA network in Tshwane, we had a LOAD of technical issues and visions and some really crazy ideas about how to get it done.

Reality, however came when we started implementing.

Joe’s post and architecture is noble. The devil, is as always in the details.

Layer2!=Ethernet

I can’t find fault with Joe’s Layer2 design. As far as it pertains to Metro-E.

Ethernet is simply the most ubiquitous protocol around, and regardless of whether the backhaul switching  incarnation is PBT, or QinQ or whatever collision domain limiting architecture the next vendor may invent, the bottom line around Joe’s Open Access network remains Ethernet.

Everyone’s got it.

How Layer2 is managed in the collision domain, and how Ethernet is trunked back to the  ISP of your choice is largely semantics and arguments around Layer2 backhaul implementations. Back when we did Tshwane, none of these technologies or standard really existed, so we simply adopted a “make things so” approach.

Limiting’s ones scope to Ethernet as Layer2 however is just not entirely viable if you’re considering to be an Open Access network operator.

Ethernet isn’t everything. The fact that I’m still maintaining one of the most feature-rich l2tpns fork’s thats now been IOT’d on everything from 3GPP PDSN’s, IPWireless INC’s  to Huawei WiMAX ASN Gateways, and GGSN’s  is a tribute to Neology  understanding of Layer2 technologies.

We’ve had do things with what we had available, what customers demand, and that has allowed us a great understanding of Layer2 access… Ethernet is but a single aspect of layer 2 access…

“ISP Preselect” is and always has been the core issue as it should be for any Open Access network that’s layered around Access, Management, and Services.

Layer2 Authentication

Joe mentioned that “To get on to the switch fabric beyond my block I use something like dot1x”.

It would be nice, in fact if 802.1x was the defacto mechanism EVERYWHERE for getting your Ethernet layer access, even if it is just “on your own block”. That would allow you to switch ISP’s at a moments notice without waiting for some “NOC Engineer” to “reconfigure” your port on the switch to now plug into ISP(a). It makes absolute sense.

Change your 802.1x credentials and BOOM, you are now trunked into ISP(b) network, rather than ISP(a)’s network.

This is generally achieved by simply slapping a few attributes onto the RADIUS  EAP-TLS/TTLS reply, and  is something that most metro switches these days  understand.

The AAA support in 802.1x means that your desktop, or notebook can  simply supply a certificate, which (gasp) will probably require  a username and password to validate the certficate. EAP-TTLS will simply require a username and password if you’ve got the AAA server cert signed bya part of your CA chain.

All of this  would allow the  Open Access network  to connect you to the service provider of your choice, via VLAN or _insert_funky_metro_backhaul_protocol_here by simpy changing your 802.1x credentials.

It also (quite neatly) allows the OpenAccess network provider and ISP to comply with some of South Africa’s niggly regulations such as RICA, that requires the loop provider, and ISP to positively identify their customer.

The problematic part itself is 802.1x and how it shits on the user experience.

The EAP Challenge

802.1x generally uses some kind of EAP based authentication protocol. So, which EAP would you like to use today ?

PEAP, EAP-TLS, EAP-MD5, EAP-TTLS, EAP-(INSERT_TLA_HERE)?

EAP, in it’s bountiful incarnations represents a number of  challenges:

Firstly, that of client-side support: Yes, Windows has it, MacOS (maybe) has it, Linux has it. If you install the right package, and twiddle the right knobs. Good luck if you’re running something odd.

Second, is that of a decent trusted PKM infrastructure, which (I’m assuming) is partly what has got Joe so excited.

TrustFabric could play a pivotal role in providing PKM infrastructure services for something such as an 8021.x based Open Access network. There’s some nice dollars to be made in that. However, people like Verisign, Thawte and others have got SAAS PKM infrastructure available as well, and have been playing that game for a long time. Even “open” platforms such as OpenID has made movements in the PKM direction.  It’s a case of “who can issue certs the easiest”.

The final and biggest challenge behind EAP and ANY kind of PKM authentication remains “getting that damned certificate installed”, and working,  and presenting a useful interface to a customer.

In other words,  something that mom and pop can use on their newly acquired Asus EE-PC and MacBook.

Good luck in trying to get people to right click on .pem, .der and other paraphernalia required to get a certificate installed on Windows.

Your best bet, unfortunately  is some kind of “client application” that makes things “easy”, such as Cisco’s or other third party supplicants. Boom, there goes most of your “plug and play” capabilities and “vendor independence”. I’d hate to see an Open Access layer2 network that requires me to authenticate with some muppets’ software that cannot run on Linux, for example.

This is aside from the fact, that the various OS’es built-in 802.1x clients will only support very specific flavours of EAP, and with _very_ specific attributes present in the cert.

The best implementation thus far is EAP-TTLS, which simply requires a username and password, as long as the AAA server is in the client’s trusted chain. This can be easily achieved by having your AAA cert signed by the big boys such as Verisign or Thawte, in a similar fashion to how SSL works.

Unfortunately it’s not supported by Windows, by default, again. EAP is a minefield. If you don’t believe me, just go read this and look at the varied support and implementations.

The bottom line is: Pick your choice of  EAP agony, or go home. Suddenly the Open Access network has become very “closed”, and getting Layer2 access has become more complicated than asking Telkom for a copper pair.

My view on joe’s Layer2: The summary

• 802.1x authenticated Layer2 access is a challenge, not just for the provider, but also the customer.
• Non-802.1x Layer2 access is an customer take-on and ISP switching maintenance nightmare.

It may sound like I’m knocking Joe’s Layer2 vision quite a bit. But I’m not. Really… It’s just the the reality of 802.1x has been so muddied with vendor specifics and “extensible” standards that it’s nigh impossible to implement 802.1x for everyone, which was the entire point behind 802.1x to start with.

The alternative, which is to provide Layer2 services without authentication, or some kind of manual provisioning or vendor-switch specific mechanism is er, well. Vendor specific, or labour intensive. It generally doesn’t allow me to dynamically choose the service provider trunk that I would like my “circuit” to be terminated in.

802.1x should have been the most elegant way to provide dynamic switching between ISP’ Layer2 access to a customer.  Except it isn’t. Maybe it will settle down some in the future. Right now, it’s a frigging nightmare. Personally, I shall wait for the battle of the EAP’s to present a winner.

I fucking hope that winner is EAP-TTLS.

Joe’s Layer3

Joe: “Anybody can sell IP traffic over this switch fabric.”

OK. How do we sell Layer3 over a switch fabric? Well, the selling part should be obvious.

Actually GETTING Layer3 access is another thing. The first step for a customer being getting an IP address.

On a generic (802.1x authenticated) Layer2 Ethernet network the client’s choice of obtaining an IP address is:

1. DHCPv4 the thing a dynamic address.
2. Statically Assign an IPV4 address via DHCP based on the MAC, or customer.
3. Have IPV6 autoconfiguration assign the thing (but hey, isn’t that the same as DHCP?)
4. Use something like PPPoE but Joe doesn’t want this ?

This is all great, and very “plug and play” if we’re talking DHCP, or IPV6 Autoconfig.

Clearly static assignments are a bad thing ™.  So let’s consider that 802.1x actually worked, and I’m trunked back into my ISP’s core, or access devices for customers.

How do I stop a nitwit from assigning an IP to himself that he’s not supposed to have?  The ISP needs a device, to do that. Something that snoops on the DHCP response/reply and makes sure that I cannot (from my MAC) use another IP address.Let’s assume that (most metro switches do this in any case).

The ISP switch, or terminate device now has to filter all traffic for my mac address to be source, and destination correct for the IP address assigned. Fair enough, that’s something that most decent metro switches can do, and the OpenAccess network operator simply trunks traffic from my Metro port to the ISP’s access switch.

Thus, there is traffic inspection required on this  device. It requires ISP switch to do L3 inspection, to enforce the all of these rules. So effectively it’s a layer3 device, or NAS, since it’s probably going to have to generate some RADIUS records as well, to account for traffic. Aside from that, in the case of static IP assignment to a customer, the device will probably also need to advertise my IP’s location to the ISP’s IGP via OSPF, or iBGP.

Dang this access device has become a router rather than a switch. In fact, it’s become a NAS in the traditional sense.

And I’ll bet that suddenly  it has all the scalability problems that a typical “NAS”, PPPoE concentrator, or other “access device” would have, due to the large amount of state, and inspection required.

Now,  let’s take it a step further. The customer doesn’t have a single IP, but perhaps an entire netblock that they want statically routed. They would also like redundancy, and load-balancing across multiple connections!

Oh god. DHCP fails miserably for that. And I’ve yet to see the device that can even implement these sort of things via a simple instruction from a AAA server, and a client device that has support for that kind of functionality.

I could carry on about the problems involved with simply using Ethernet as a DHCP or IP platform but I’d have to write another 50 posts. The bottom line is that the basic Layer3 situation mentioned here has some serious limitations.

We need something else, something that works, something that does more than just handle a single IP, and a single user.

PPP is not dead

PPP is sometimes referred to as “legacy”. In fact, PPP was RFC’ed in 1994.

My definition of “legacy” is “it fucking works”.

Whether it’s PPP over Ethernet, or PPP over GRE (pptp) or PPP over L2TP, or PPP over SoNet is irrelevant.

PPP as a baseline protocol has solved so many networking problems and supports so many features that “modern” ideas of network simply doesn’t support.

Taking enterprise Layer3 access principles and applying them to “circuits” for customers is like trying to take over the entire world with hotspots, walled gardens and DHCP.

PPP is not dead. It supports IPV6, Routed netblocks, bridging (can you say dynamic load-balanced VPLS?) , encrypted authentication, encrypted data transfer, load balancing, bonded, or multiple links. The list just goes on. Every single circuit based networking problem in the past two decades has in some sense been solved with PPP.

Windows, Linux, MacOS and a plethora of other operating systems all support PPPoE as a built-in. Every home “WAN” router I’ve seen supports PPPoE. Very few of them supports 802.1x. Even the most menial Cisco router supports PPPoE.

By the device count on just the above mentioned vendors, PPPoE certainly has a damned good application still in the real world. A market that cannot be ignored, not just from a pervasiveness perspective, but also from a functional perspective.

Layer2 “Pre-select”

The true  challenge for an Open Access network, is to allow the customer to do carrier “pre-select” for his Layer2 Service. This effectively gives them a “circuit” to their provider of choice.

This is what an Open Access network should provide.  “Layer2 Preselect”. Take my frigging circuit and connect it to the carrier of my choice.

Joe’s case for the 802.1x and L3 DHCP style service is certainly one use-case, but it’s certainly something that is applicable to a general road warrior or home user. It’s based on Ethernet, and IP over Ethernet. That has it’s limitations.

The fact is that PPP is also a Layer2 protocol. PPPoE should be the second defacto Layer2 service provided by any Ethernet-based Open Access Network. To not do so would be suicidal from a business perspective. An OA network design should provide for PPP based “Carrier Preselect” as well as Ethernet based “Carrier Preselect”.

PPP is used on 3G/HSDPA networks, CDMA networks. These are “legacies” that one has to contend with in an Open Access network. Because nice as it may be to dream about, Ethernet isn’t yet everywhere, and there are simply many networking technologies where it doesn’t make sense.

Open Access networks MUST allow for Layer2 preselect. Period. From there on, the implementation should be left to the ISP. The preselect should cater for IP over Ethernet, or IP over PPP. Those are the two most pervasive technologies around.

How to implement PPP over Ethernet?

One of the many possible solutions is fairly simple to implement.

The default “unauthenticated” VLAN for any Metro-E switch that the Open Access network operates trunks through to every ISP that offers services on the OA network. ACL’s are configured to ensure that only PPPoE packets are allowed through this default VLAN, and only to the know MAC’s of the ISP’s PPPoE concentrators. In order to scale it, certain segments of the citywide Metro-E network are trunked back to the provider on different “circuits” or VLANS where they can decide to implement one BIG PPPoE concentrator, or many smaller ones.

Each ISP advertises it’s AC (PPPoE access concentrator) on this default VLAN. A customer wishing to use a specific ISP specifies his username, password, and AC-name, associated with the ISP.

He terminates on the ISP’s PPPoE AC. The ISP pays a per megabit rate for access to this VLAN to the OA operator.

To handle  Joe’s case, the Metro-E switches implement 802.1x. All 802.1X AAA requests are forwarded to the Open Access operators’ AAA servers, which makes a decision to forward the request to the ISP’s AAA based on the outer unencrypted anonymous identity of the Access-Request (which normally still contains the ISP’s domain)

The Open Access operator’s AAA forwards the Access-Request (and ensuing EAP conversations) to the ISP’s AAA, which authenticates the user and reply’s with an Access-Accept to place the user in the ISP’s “service vlan”.

The client DHCP’s and address and get’s his service.

This is but one possible implementation (on the PPPoE side) there are many other options including PPPoE proxy’s, relays, tunneling etc.

QED

An open-access network should provide and support as many industry standards as are possible. Simply providing Ethernet across a nation or Metro is not the entire picture of the solution. One has to consider all the possible use-cases, technologies, supported standards, nice-to-have standards, and their viability.

The fact is that a RADIUS authenticated username and password remains the simplest, most commonly supported standard across a whole range of technologies. How you “make it so” is the differentiator.