System 1 and System 2

What is this ’system 1’ and ’system 2’ anyways?

System 1 and system 2 have been referred to by many things, and system numerology isn’t the best nor the first name for them. Some texts refer to them as monkey brain and machine brain (The Chimp Paradox).

The best way to view ‘the system’ (as proposed by Keith Stanovich and Richard West), is system 1 is your reactive part of your brain, and system 2 is called into action when there is no known or available response in system 1.

Over time you and your environment train your system 1 to react to give situations and stimuli. Whereas system 2 is more like your long term memory and stores key facts and information … etc. The two things to remember are;

1. System 1 is effectively ‘in charge’. There’s no decision or reaction that doesn’t initially engage system 1 first (aka ‘the chimp’).

2. Our systems/brain is lazy, therefore if system 2 needs to be engaged, it will work with system 1 to see if there is a ‘good enough’ response, even if that response doesn’t completely fit what is actually needed (aka ‘the computer’).

As per the old cliche, wherever possible ‘think twice and cut once’ (aka ‘the human/ you’).

Negative Positivity

Over recent years we humans have focused on positivity being ‘the cure’ for various problems and issues, whether it be our lives both at home or in business. However, more recently this has started to spillover into what I call, negative positivity. And by that I mean, positivity at all costs … which ironically comes with quite a toll to be paid.

The human race isn’t designed to be happy all of the time, and coping with this thing we call life, is hard enough in these modern times with 24×7 media, news coverage and operating cycles, so to be positive 24×7 is not natural.

We need to be able to flow unabated between happy, sad, angry, elated, stoic … etc., without fear of retribution from our fellow human beings. Which is compounded by the endless stream of digital content on social media, ‘showing’ everyone being happy and successful ALL of the time.

Just remember, what others show you isn’t always the truth and be conscious of what your chimp is interpreting for you (just remember to put it through your own ‘computer’/ filter).

Media

Social media and the wider media and advertisements, are just that, media. And whether you like it or not, whilst there is a lot of respectable outlets and sources, it is peppered with additional information and sources which want to hold a false mirror to you for their own gain.

The biggest underpin of the social element of that is companies that want to sell you the idea that, whatever idea they can impart to you or mental thread they can pull on, that they and only they, have the answer. And a lot of the time this is done with either stereotyping or negative re-enforcement.

The second is individuals selling their story of how perfect or amazing their life and/ or achievements are. There’s several articles every year about how filters and photo editing on social media portray the subjects as ‘perfect’ or ‘beyond human’, and then the ‘scandals’ as a celebrity is caught out faking their beautiful life to the masses.

My personal opinion regarding social media content, that with the exception of genuine people (usually family and friends) and fact checked news outlets, the wider influencers and media that they generate (videos promoting the ‘latest’ skin care or cure for loneliness) is all fake and ultimately as per YoYo’s in the ‘00’s its all temporary. They prey on people’s insecurities and fear of missing out (FOMO) to create peer pressure (either externally or self inflicted) … it’s one big algorithm and it’s stacked against … you!

And in typical human fashion of keeping up appearances or FOMO, we can push ourselves into situations which can have long term negative consequences.

What you have however, is what you’ve achieved and accomplished for your lifetime. No-one can take that away from you, and if someone is using any of this (or the lack of it) to be rude to you, that’s there problem, not yours. What you can do, (is kind of what I’m doing now) is share your experiences, learnings, failures … etc., so that others can learn from them to improve themselves in a positive way and rebuild their resilience that the modern world has slowly stripped them of.

I’ll explain later how FOMO for me nearly put me in the hospital.

‘Coping’ Mechanisms → Tool Kits

‘Coping’ is a cliched term, so I’d like to move the conversation to ‘Tool Kits’. Without repeating several of the suggestions that appear throughout this article, the one that should be advocated here the most is awareness. Once you’re aware you can act, and remember what worked one time or for a specific scenario might not work for the current one; hence needing a tool kit in order to be more flexible as life changes and evolves.

The place I started was getting comfortable in my own skin, or at least as comfortable as you can. It’s easier said than done, and depending on the situation I’m in, I can quickly find myself regressing if I’m not care. I’m still working my way along this journey, and so far it’s at least yielded the results in the ‘Searching for Success’ series.

References

Recommended Reading/Book(s)

• https://chimpmanagement.com/books-by-professor-steve-peters/the-chimp-paradox/

• https://www.amazon.co.uk/Thinking-Fast-Slow-Daniel-Kahneman/dp/0141033576

I’m a network engineer by trade and not a developer, DevOPS, platform engineer and/ or a data engineer/ DBA, therefore the thoughts below are moving towards a vision/ utopian state, rather than best practices.

The Value Proposition

One of the biggest hurdles within an organisation when on a “data transformation” journey is the cost of storing the data. Whether the data is a rest, hot, warm, lukewarm, cold or freezing/ frozen, it’s a KB here and a KB there of data which soon adds up.

However in a data driven world data is king/ queen/ $insert deity of choice$. Without it a modern network operator is blind to what is happening in their network, and so there’s unexplainable outages, missed opportunities for proactive intervention, and overall a poor customer experience.

So how do you turn that data into something valuable? Well unlike social networks or cookie scrapers for telecommunications providers, I see the following top three as the drivers/ KPIs for any business case:

Customer Experience/ Satisfaction

A happy customer in telco is a quiet customer. And other than consuming connectivity, therefore a fractal of last mile equipment, an even smaller fractal of a local point-of-presence (PoP), a minute fractal of subscriber termination, backhaul, peering and transit; the total cost of keeping the customer is relatively low.

The upside of this, is depending on how “active” in any community they are, there’s free PR; “Hey $TELCO$ is great, the service just works and …”, and (as long as there’s no shocking price hikes (Q4 2025 O2 Price Hike), they’ll stay a subscriber of the service until either performance/ experience drops or the value (typically price point) of another service becomes more attractive.

Mean Time to Innocence (MTTI) or not

The one thing I’ve learnt in my years of experience operating networks, the faster you can prove the innocence or not, of a network, the faster the problem is resolved.

And with modern networks getting less complicated technologically, but more ‘noisy’ data wise in regards the number of customers using the network, that means traditional ways of monitoring the network and the logging of the data have become obsolete.

And the data that drives the MTTI, is one of the linchpins of keeping customer experience/ satisfaction high for using your network.

Cost to Serve

The ‘elephant in the room’ if you will, is the ‘ugly’ topic of the costs. There's the costs to get a connection to a customer, then the costs of ‘acquiring’ the customer (yes for those who don’t deal with the commercial/ product side of a business, that is a term), the costs of maintaining the customer, and finally the costs of disconnecting the customer; the customer lifecycle.

Connecting and Acquisition

Whilst data has limited impact on connecting and acquiring the customer, other than the best and most economical route to get to said customer and what package would best suit and be attractive to sad customer, the major drivers for those costs are time and resources.

So the better the data is about what is needed to connect a customer and the time needed to achieve a good quality job, or about what demographics are in the area to offer targets deals either based on their preferences or time of year, i.e., “Hey $INSERT_NAME$, you’ve had multiple WiFi faults this year, would you like to upgrade to the ‘whole home' WiFi’ package a Z% discount rate?” or “As $INSERT_GAME$ is sending out $UPDATE_SIZE$ patch on the $INSERT_DATE$ would you like a speed boost to $INSERT_TOP_PACKAGE$ for 12hrs to make sure you can get the game as fast as possible?”

Maintaining the Customer

As described above in Mean Time to Innocence (MTTI) or not, the data is king/ queen/ $insert deity of choice$.

So imagine (if you will) gaining the ability to be proactive and turn the contact centre into a revenue generating position of a business, rather than a cost overhead.

What do I mean by this? Well turning the ‘flow’ (if you will) of how a contact centre engages with the customer base from “Hello, my name is X, how can I help you today?”, to “Hello, my name is A from BCORP and we suspect that there might be a fault due to occur on your service/ in your area, would it be ok for BCORP to book an appointment with you so that C can visit you to do proactive work?”.

Alternatively in a reactive situation, messaging the customer BEFORE they try and call in 😱🤯 “Hello, this is BCORP we have notice a problem with your service, please see $URL$ for live updates. We will message you again in X minutes/ hours (depending on SLA) with an update”. And in said updates, being able to have clear and concise, data centric updates from the data driven insights (not marketing fluff).

Now that might not be the exact method or style of engagement (I’m no marketeer or public relations guru, I’m more of a ‘networks guy’), however the ‘power’ of getting ahead of something builds confidence and trust in the customer base.

That’s All Very Nice, However How Can This Be Done?

So with the skeleton of WHY above, and some of the WHAT covered in my previous post 🤖 “I have detailed files” 🤖, the HOW starts to form based on 🎶 Do you want to find a Network? 🎶.

What Does The Ecosystem Look Like?

Strangely enough we’re going to start embarking on a fairly familiar journey, automation. The difference is, this is focused on WHAT can be done and some direction/ options on HOW to do it (not every option fits every organisation), rather than dry prescriptive ‘play nicely with each other’ “standards”, and finally a utopian endgame which I hope we can see in my lifetime, autonomics.

Drinking From The Firehose

So as stated previously, modern networks are generating more and more data “noise” and this is due to increasing customers taking the service, and observability required to maintain the service (or a developer leaving the debugging turned on and pushing to the live/ deployed code base 🤦🏻).

Now the ‘knee jerk’ option is to start filtering, however I recommend collecting all of the data and keeping it as long as needed whilst observing the legal boundaries that you have to operate your network(s) in.

Doing this brings it’s own challenges, fundamentally the legal requirements and cost, hence me starting this post with the WHY statement.

Where Will the Data Come From? “Data Source”

Data is generated within a system, closed or open.

For networking the common ways to communicate the telemetry data (streaming telemetry/ dial-out) is via; RADIUS, Syslog, SNMP traps, gNMI, NETCONF, IPFix and several others (including vendor proprietary). There are also dial-in options such as SNMP, gNMI and NETCONF. Typically these pieces of data are stored in a database or a log file with limited or no API. Therefore, an API is needed.

For systems/ software these tend to be dial-out options, whereby the ‘north’ system makes a ‘southbound’ request to the supplicant system via an API, i.e., an Operations Support Systems (OSS) stack will make a provisioning request via API to the Network Management System (NMS) or Element Management System (EMS). Typically this is a API platform with a WebUI, (until the humans are no longer needed).

The foundational block of this, is to build data products and build them per role type, and the ingest data products should be laid out the same way that the Implementation/ Intent Engine products are, to help maintain the Imperatives (inside the Policy Engine), however more about that later.

Overall as there are several sources of disparate data, so building relationships between them is critical to the longterm success, especially as the entropy of the data grows.

How to Collect the Data? “Ingest Engine”

As there’s so many sources of data, and in an operational network a lot of outcomes from the data are only needed in and around the area that it was generated, then using local caches/ agents (see Autonomic Service Agents (ASAs)) is a way forward.

This also means that local processing is required and anything:

1. outside of the:

   1. compute capabilities.

   2. current interaction sets/ training models.

2. which is needed globally (i.e., RADIUS, event alarms).

Should then be processed at a central location, before being returned to the local store to be actioned in the subsequent engines.

Once processed events can be sent back to a central location for wider viewing and understanding.

How to Store the Data? “Ingest Engine”

Now this is where the nuances start.

So, an example of this is say, an Optical Line Termination (OLT) device. An OLT will generate data for the:

• User Network Interface (UNI) port on an Optical Network Terminal (ONT) unit

  • Throughout

  • Speed of the interface

  • Duplex of the interface

  • CRC errors

  • … etc.

• The light levels of the PON port of the ONT

• The light levels of the PON port of the OLT

• … etc. (see 🤖 “I have detailed files” 🤖, for more info)

For the most part it will generate that information in a standards based way, although the units of the output might vary, i.e., interface speeds are now in the Gbps, however for some vendors when the code for the Network Operating System (NOS) was created the speeds were in the Kbps or Mbps, so the units needs refactoring.

However when it comes to the logging of its events bus, that’s when things go proprietary, and normalisation in my opinion is the best way to deal with this at scale. Otherwise you’re such with a large matrix of event triggers which can become cumbersome. Whereas normalisation can follow an If This, Then That (ITTT) approach, which is more logical for a human to backtrace.

Finally you need to ‘join the dots’. Ultimately there should be a data product per element with a view using the profile applied based on its role in the network. However as services are provisioned over said network element, the service will need a relationship creating with the element(s), based on the resource(s) it consumes from said element(s). There’s a few ‘schools of though’ on this, and whilst as I said above I’m not a DBA, I personally visualise this as a 3D graph/ model, using nodes to set the anchor(s) and the values are then associated with the nodes. This is so a value can change (such as an IP address, VLAN ID, interface number … etc.), and not break the relationship; unless the node needs to be removed/ deleted, i.e., the line card is removed.

Example graph relationships

Normalising with Data Privacy

Whilst this is easier said than done, I believe that aiming for a normalised view of (or at least a way to view) the data with data privacy at the heart will break the back of this problem. The reality, is vendors of hardware and software have little or no incentive to do this, never mind the quagmire of trying to inflict this via a standards body.

Ok, so how can this be achieved? One option is to use Large Language Models (LLMs) and train it on key word analysis on a per vendor per solution in order to gain insights and analysis. If this is done in conjunction with a Data Mesh architecture, it means the next step of joined up thinking (finding the signal in the noise), can begin.

Replication and Redundancy

So the likes of the hyperscalers and content distribution networks have grappled with this for some time and the likes of Netflix have published several articles which are worth looking into. I don’t fully understand the HOW parts of these articles, however the WHAT is simple; data needs to be replicated, as disparate systems need access to the same information, however said data needs to be clean, uncorrupted and the same amongst the disparate systems.

Introducing Conflict-free Replicated Data Type (CRDT).

Building a Resilient Data Platform with Write-Ahead Log at Netflix

Rethinking Netflix’s Edge Load Balancing

Additional reading

Load is not what you should balance: Introducing Prequal

Implementing the Dots “Intent Engine”

This is where modelling is very important and creating frameworks that are vendor agnostic and data driven.

What I mean by this is, historically vendors have created their own way of laying out and storing the data which drives their network element(s), however when you have several vendors in your ecosystem, keeping data models mapped per vendor becomes unsustainable, especially when the vendor changes their models between NOS versions.

So as explored above, keeping your own data model which can be abstracted per vendor is how you can drive your intended outcomes (more on “Implementation Engine” shortly).

What the F*&k is an Intent!?

Intents in my world are finite actions, i.e., “I would like to eat a tasty bowl of rice”, in the realms of reality, for that term either; the bowl will become empty or my stomach will become full.

In networking a finite request of a network element to perform a task would be:

• provision network element X with serial number Y for role Z with profile 1234.

• add feature X to network element Y from updated profile 4567.

• provision customer X for product Y over path Z starting a date A time B and finishing at date C time D.

🎶 Woop-woop, that's the sound of da police 🎶 “Policy Engine”

Policies are what you would expect, the ‘policing’ of the Intents. They should be set by the business/ organisation objectives and follow an interpretation of Asimov’s Three Laws of Robotics:

1. A Network Ecosystem may not injure the network or, through inaction, allow the network to come to harm.

2. A Network Ecosystem must obey the orders given it by human beings, except where such orders would conflict with the First Law.

3. A Network Ecosystem must protect its own existence as long as such protection does not conflict with the First or Second Law.

So if an Intent is finite, then you’ve guessed it, a Policy is infinite. Using the boundaries set by the business/ organisation logic and the telemetry from the Ingest Engine to measure the KPIs/ metrics/ data coming from the network elements, it is the binding ‘force’ behind keeping a Network Ecosystem stable and running as per the require Intent(s) of the Network Operator.

That Cute, However What Are My Options? “Implementation Engine”

If creating your own data structures, and relationships is not something you want to do, there are quite a few tools out there now. One I would recommend exploring is NetBox (no I don’t work for them, and no this isn’t a paid endorsement).

NetBox has build the relationship between networking elements, circuits, IP address, rack layouts … etc., all for you in one hand tool. They have also extended it’s abilities with APIs and the community have created various plugins, MCPs … etc., which with a Policy Engine (I’ll come onto that next), will start stitching a lot of this together for you, example https://community.juniper.net/blogs/victor-ganjian/2025/11/01/network-automation-with-ai-and-junos-mcp-server.

Summary

Balancing the requirements of a business/ organisation, against the demands (or the perceived ones) of the customer/ end-user has always been a constant struggle, as the business/ organisation wants to be at the top of the listings for its market (shareholders/ return on investment), and the customer/ end-user wants to pay the least amount for the best service possible.

So leveraging data to drive towards an autonomous future, helps (within reason) improve good with constant feedback, if scaled correctly reduce cost by not wasting time/ duplication of effort/ systems … etc., and finally improve speed for the mundane routines1.

Overall, none of this is easy, however most of this is automating mundane tasks, speeding up data analysis.

One of the biggest headaches an operator faces on a daily, even minute by minute basis is, ‘What happened in my network?’, ‘What was it like before?’, ‘What changed?”, “Who made a change?”, “When was the change made?” … etc.’.

And so leveraging a Data Mesh can support proactive analysis, as well as reactive alerts, alarms, whilst powering/ super charging automation … etc., for the likes of network monitoring, network management, event correlation and outcomes … etc.

This article wont get into ⛱ Datamesh or ❓What is a Data Mesh, however this blog will outline how the information coming from networking devices/ elements into a Data Mesh can be used and how disparate data can be used together in order to drive real outcomes.

It becomes self-aware at 2:14 a.m. Eastern time, August 29th

So where to start? On the basis that telemetry from the networking devices/ elements is already being captured in some form of collector and/ or data storage/ Data Mesh. Then if you’re sitting comfortably, then I will begin.

Terms of Reference

Before we dive too deep too quickly, here’s a quick recap of some terminology used regarding the hardware of a network device/ element.

So taking something like the ufiSpace S9510-28DC:

It it composed of the following elements:

• Routing Engine - Intel Denverton-NS 4-Core @ 1.6GHz (Standard) or Intel Denverton-NS 8-Core @ 1.7GHz (Premium)

• ASIC (Application-Specific Integrated Circuit) - Broadcom Qumran2a BCM88483

  • SerDes (serializer/deserializer) - 16 x 50G + 36 x 25G

Layout

What this will logically look like inside the ‘chassis’ (even if it is 1RU):

Routing Engine (RE) <> Packet Forward Engine (PFE) Relationship

The RE is the ‘brain’ of a networking element, it holds and processes all of the information that is given to the networking element, running configuration, firewall filters, QoS … etc., and the Routing Information Base (RIB).

The PFE is the ‘executer’ when it comes to decisions about where to route traffic/ packets. It is programmed by the RE, with the best available routes from the RIB to create the Forwarding Information Base (FIB).

Optical and SerDes

In order to support the port speeds, the SerDes bandwidth needs to equal or be higher than the optical interface, i.e.,

• 400GbE optical interface = 8x 50Gbps SerDes

• 100GbE optical interface = 4x 25Gbps SerDes

“Lasers”

So now it’s time to discuss light, wavelength (frequencies) and “lasers”.

All optical transceivers have a transmit (Tx) and receive (Rx), they usually use separate lasers and fibres to keep the optics separate, as the wavelength is the same for both the Tx and Rx:

However in BiDirectional (BiDi) optics, there is only one laser, and therefore one fibre strand is needed, and so different wavelengths are used for the Tx and Rx.

Note that the colours on the release handles of the transceivers delineate what the Tx wavelength is:

• yellow = Tx 1550nm and Rx 1490nm

• purple = Tx 1490nm and Rx 1550nm

Are You Sitting Comfortably? Then I’ll Begin

The importance of this morning and the relationship(s) between them is critical when operating modern networks.

The size and scale of modern infrastructure is now reaching the point that disaggregating to reduce the blast radius is becoming the norm, and so this increases the surface area that needs to be managed. Therefore, the number of network elements/ devices is always increasing.

So in order to trace a fault, or understand a change/ or impact … etc., understanding the relationship(s) between network elements/ devices and the effects they have on each other drives the operational excellence requirements of high quality monitoring and data collection. This is so any trend can be called out, any proactive intervention can be taken, any reactive troubleshooting can immediately identify the root cause and any investigation can have a play back history.

Here are some examples of how the data exhaust from these elements can be used.

CPU

PFE CPU is spiking and/ or is high and holding constant, however the RE CPU is low

This can mean that there is some large routing updates being programmed and or large amounts of traffic are being sent through this PFE. If this scenario continues to any period of time then the PFE might become overwhelmed and start dropping packets.

The RE is spiking and/ or is high and holding constant, however the RE CPU is low

This could be due to a large amount of routing updates occurring in the network, and as such this behaviour will be seen on many devices or there could be something local to the device and causing a large amount of logs to be generated. The slower the CPU the longer the updates will take, and if the updates continue to flood (i.e., a link(s) flapping), then smaller devices can get overwhelmed.

Memory

Interfaces

Examples

The following are examples where the light levels were previously fine and had a ‘lit’ and working service(s) over the interface(s).

Rx Light Dropping

This can be caused by problems on the fibre(s) between the devices, to the Tx laser on the adjoining device (whether that is an optical shelf or a network router … etc.).

Trend data is very important here as well as the light levels of the device physically attached to it, i.e., if there is an optical chassis for long distance networks (LDN), i.e., are the levels trending down over time, or just fluctuating?

• If a downward trend, have move services been added to the fibre path?

• If fluctuating, are there any ground works in the area? i.e, giant yellow fibre finders.

Tx and Rx Light is Dropping

This can be quickly narrowed down to, is it just one port, or are others affected (see and how some ports share PHY on the ASIC).

• If it is a single port, then what is the age of the optical transceiver?

• If it’s the adjacent ports sharing the PHY , are they all affected to the same magnitude and at the same time? i.e., lack of power, and or increasing temperature; as the higher the bandwidth and the further the distance a transceiver can operator over, the more heat and power it creates and needs.

Temperature

The Inlet Temperature is Rising, However the Environmental Temperature Isn’t

This could be a symptom of the environment not managing to clear away the hot/ warm air produced by the network element/ device.

The Fan Speed is Increasing/ Fluctuating, However the Environmental Temperature Isn’t

This can be caused by a few factors:

• The network element/ device is under load, and the CPUs are running hot, so the system increases the fan speeds to clear the air off the CPU heatsink(s).

• The bearings in the fan could be failing, causing the fan to spin more freely/ faster.

• The solenoid/ sensor could be failing, causing the fans to spin erratically.

Power

Additional References

Inlet Temperature Ratings

Telco Vendors

c-Temp

Commercial Temperature

• NEBS 0 to 45*C

• DC NEBS up to 40*C only

e-Temp

Extended Temperature

• 0-55C (class3.3)

i-Temp

Industrial Temperature

• ETSI Class 3.4 up to 70C

• NEBS GR63 and GR3108

• All parts of the router must be rated for these conditions

• For example: PFE component designed to operate up to 110*C

• More expensive parts

• Smaller heatsink and less fan speed: less power

Industry (including Optics) broader standards

• Commercial: 0 °C to 70 °C (32 to 158 °F)

• Industrial: −40 °C to 85 °C (−40 to 185 °F)

• Military: −55 °C to 125 °C (−67 to 257 °F)

So Where Do We Go From Here?

Well if you’ve made it this far, well done. The answer you’re looking for is … Data Mesh.

Now as per the pre-tence, whilst I won’t get into the details of Data Mesh, the principal is; gather data in the most efficient way for it, and for your organisation, then build the relationships between the data/ nodes that are right for your organisation. Graph Databases/ GraphQL are great for this, however scaling and experience of Graph will play a big role in how you deploy this.

Granted the more network elements/ devices you have, and the more data points you collect the larger the data storage you’ll need and the better data retention policy you’ll need for ‘hot data’ (instant), ‘warm data’ (the recent period) and ‘cold data’ (long term).

One of the more ‘modern’ concepts that is used predominantly amongst marketing of a technology businesses, is what network engineers have been using for decades to separate the ‘core’ Interior Gateway Protocol (IGP) ‘the underlay’, from the ‘services’ that need to be supported over the top of it, ‘the overlay’.

So What is an Underlay?

An underlay is the key protocol(s) which connect the routers and devices to the network (the Autonomous System Number (ASN)), and allow routing to occur within the network, i.e., exchanging loopback information of the service routers/ devices, example for Pseudowire Headend Termination (PWHT).

This is usually an IGP such as IS-IS or OSPF, however it can be MPLS, Segment Routing (SR), MPLS-SR, Flex Algo … etc.

Overall the Underlay is usually split into ‘areas’/ regions based on either technical limitations and/ or business requirements with the key driver to reduce ‘blast radii’ should anything go wrong in the ASN.

Example: There would be several regions for a tier1 operator, the diagram below is quick image to show the relationships between the boundaries.

Example from Metro Fabric and Broadband Edge—Juniper Validated Design (JVD)

Metro

This is typically the ‘area’ which covers the ‘last mile’ (the connections to the subscribers/ end-users including splitters, street cabinets … etc.), and is all physically connected back to a central location in a hub and spoke topology.

If the Metro is located inside say a city, then there will be multiple Metro’s (as needed), to connect the subscribers/ end-users, just like postcodes.

Usually the Metro Headend will live somewhere central to the deployment, and for many years in the UK, since the 2000’s telephone exchanges have been used (see Local Loop Unbundling).

Unfortunately, with the closure of the exchanges in the UK, this means that UK operators/ ISPs need to invest in facilities of their own, example CityFibre build FEX’s.

Metro Aggregation (AGG)/ Headend Aggregation (AGG)

This is where multiple Metro’s are connected to an aggregation point. This will facilitate connectivity between the Metro and the Region. And if required house some of the service termination devices, i.e., Broadband Network Gateway (BNG)/ Carrier Grade Network Address Translation (CG-NAT), NNI handoff (A10-NSP), … etc.

This can be a city level AGG, and requires a high degree of resilience and redundancy both environmentally, and connectivity (both downstream to the Metro and upstream to the Region).

Region

This covers a geographical area and is the next level of aggregation for the Metro AGGs. This will typically follow a country’s geographical mapping, as it will depend on the literal ‘lay of the land’.

Core

This is where the ‘magic’ happens, as this comprises of the highest bandwidth links, dense optical equipment, and other than upgrading capacity and devices, should be the lowest touch point in any network (that’s not to say that several jobs aren’t carried on in any one day/ week, however it should have less adds, moves and/ or changes (CRUD) compared to the other ‘areas’.

So if that’s an Underlay, What’s is an Overlay?

An Overlay is the service, it’s the layer that is used to transport the subscriber/ end-users service from the first device which can create the overlay (usually a Provider Edge Router (PE router) or Customer Edge Router (CE) in the case of SDN), to the service termination device (usually a BNG or an A10-NSP).

To do this, you need to build either:

Pseudowire(s)

A pseudowire(s) can we be created with:

• L2Circuits.

• VPLS VPWS.

• EVPN VPWS (E-LINE).

However, they require the endpoints to be distributed via the ‘underlay’, i.e., loopback IPs via the IGP.

Example EVPN-VPWS (E-LINE) from Metro Fabric and Broadband Edge—Juniper Validated Design (JVD)

MP-BGP VPN(s)

Multiprotocol BGP (MP-BGP) can be used to carry several protocols to create L2 or L3 VPN:

• VPLS.

• EVPN E-LAN.

• VXLAN.

However, they require the endpoints to be distributed via the ‘underlay’, i.e., loopback IPs via the IGP.

SDWAN(s)

Software Defined Wide Area Networking were (for me) what popularised the term ‘overlay’, as they automatically build VPN tunnels based on policies and controllers.

By using a controller to coordinate the endpoints and build the ‘overlay’ of tunnels ‘on-demand’. they remove any dependancy on the ‘underlay’.

Summary

Whilst pseudowire(s) and MP-BGP have a dependancy on the ‘underlay’, the benefit comes from not needing a 3rd party controller, as per SDWAN, which is unaware of the topology.

SDWAN compensates for this by using jitter and latency between the end-points to ‘monitor’ the underlay and where necessary steer traffic, however this only works on a point-to-point basis, whereas pseudowire(s) and MP-BGP can leverage the IGP which is calculated link-by-link/ hop-by-hop.

In any network engineers career comes the time you need to adopt and/ or operate a network which has already been created. And before you can work on capacity vs. utilisation, you must first achieve enlightenment around what devices are in the network, and what relationship do they have to the other devices and elements in the network.

So putting my best singing voice on and in the tune of Yentl/ Frozen I respond 🎶 Yes, I want to find a Network? 🎶

What Are the Main Problems/ Pain Points?

• The detail of the information at the right level of the documentation.

• The ease of interrogating said data.

• And the validity of the data stored in the various repositories.

Examples

Device Upgrades

A device at the edge of the network gets upgraded either to a new model or a whole new vendor, if the documentation isn’t updated then:

• engineers will be confused either when the attend site and can’t easily find the device (assuming device labels are still in size 8 font due to lack of facia space).

• when the log on to it remotely to troubleshoot or configure a new service.

• the automation breaks causing services not to be provisioned and if there’s no monitoring this could be broken for a while.

Maintenances and Change Control

Without device to device relationship knowledge and/ or provisioned service, working out which services could be impacted by a maintenance under change control get’s real difficult real fast.

And in a post incident review, it can get very messy working this out.

Where to Start?

Most organisations will have a network diagram, which will at least cover the PoP to PoP connectivity or relationships, however these will predominately be commercial or marketing focused (or curated), so for a network engineer, this has limited value and to a network operator, if fairly useless, other than knowing which facilities to go a inspect (or look for the bill from).

Example: PCCW/Console Connect

Looks pretty, however it only offers up information regarding city to city connectivity, not which PoPs within said city connect to each other … etc.

The Devil is in the Detail(s)

Depending on the size of a network, there will usually be diagrams showing the following layers:

• World

• Continent (if required)

• Region, Theatre and/ or Territory

• PoP

• Rack

• Cabinet (say in the street)

• Duct

However keeping the detail(s) of this information up-to-date is tricky, so the diagrams will hopefully show:

• PoP/ Device hostname (maybe a vendor logo).

• Link speeds (usually by using colours and ownership).

• PoP/ Device to Device relationships/ links.

Another repository is the CMBD/ inventory management system, as at a minimum IP addresses, cross connects, circuits and device locations should be recorded here for basic NOC operations.

However without automation and audits, this repository of information can quickly fall out of sync with what is deployed in a network.

Example: Netbox

So How Do We Find the Detail?

The only ‘source of truth’ is the network itself, so the network needs to be interrogated and using the following guidance.

Guidance

Find out the following for a single device that you have access to:

• What is the device?

  • Vendor

  • Model

  • NOS

  • hostname (the one actually configured on the device)

• What is its role? Provider Edge (PE), Broadband Network Gateway (BNG), P-Node (Core), Aggregation … etc.

• What IP addresses does it have?

  • Loopbacks

  • Physical Interface Address(es) (IFD)

  • Logical Interface Address(es) (ILF)

• What routing protocols is it running?

  • What are it’s neighbours? (yes the routing protocols want to be good friends).

• How many Address Resolution Protocol (ARP) entries are there on the interface?

• What is the description on the interface?

  • If an IFL what is the relationship of the description to the description on the IFD?

Parameters and Relationships

So lets say, you’ve logged into a device and you have most of the information above, what do the values of those outputs give you and what parameters can be applied to start working out the next move to discover the network?

What is the Device <> What is it’s role

So you’ve found out that the device is say a Juniper ACX or a Nokia 7200 series and its hostname is pe01.locationX and by applying some logic we can start to ascertain that specific vendors models abilities and draw out first conclusion based on the following::

Storage

Create an Interface record to the Network Knowledge Graph against the device entry.

Additional Data Required?

• Vendor Knowledge Graph, i.e., key scaling information, and supported features on a per vendor/ model basis.

• Current business architecture rules, i.e., an ACX =/= a P-Node.

Parameters

• hostname seems to indicate a PE role.

• the vendor model capabilities mean that it is NOT capable of being a BNG.

• the scale of the network and/ or the business logic means that is NOT capable of being and/ or will never be used for a P-Node.

Conclusion

High probability of being a PE.

IP Addresses <> Device Neighbourship

Whilst in networking, when devices form neighbourships, they’re not always “good friends” (see EANTC interoperability test reports), the data point and the values are key to understanding is the link point-to-point or a wider layer 2 network (LAN/ WAN … etc.).

So you’ve found that one of the interfaces IFD or IFL is a IPv4 /31 and IPv6 /127, and the interface is UP. Next you find that there is only 1x ARP entry and/ or 2x IPv6 neighbours on the IFD or IFL, we can now draw a second conclusion based on:

Storage

Create an IP record to the Network Knowledge Graph against the Interface entry.

Additional Data Required?

CMBD IPAM.

Parameters

• IPv4 /31 and IPv6 /127 indicate a point-to-point link.

• 1x ARP and 2x IPv6 neighbours entries (1x entry for the configured /127 and 1x entry for the automatically generated private address) indicate a point-to-point link.

Conclusion

There are no other devices on this interface.

Description <> Device Neighbourship (with an IP address(es))

So you’ve found that one of the interfaces IFD or IFL has a unique description, there’s an IP address and the interface is UP. The description might be something like ‘P2P CR01 <> CR02’ or ‘LINX LAN’ … etc., so with the IP Address the usage for the interface can be drawn and/ or cross checked.

Storage

Create an Description record to the Network Knowledge Graph against the Interface entry.

Additional Data Required?

• CMBD IPAM.

• CRM/ CMDB circuit list.

Parameters

Apply the necessary business logic and correspond that with the IP address stored for that interface in the Network Knowledge Graph to determine the interfaces usage.

• CR indicates the interface could be a core link.

• LINX indicates indicates the interfaces could be a public peering LAN.

• POxx or AExx usually indicates a port-group or aggregated ethernet interface.

Conclusion

Subject to business logic, however it should not be used as a deciding factor for the interfaces usage, i.e., the interface might say ‘Office LAN’, however the IP address is a public one assigned to a public peering LAN.

Description <> Device Neighbourship (without an IP address(es))

So you’ve found that one of the interfaces IFD or IFL has a unique description, however whilst there is no IP Address assigned, the interface is UP.

The next thing to look for is whether there is any layer 2 configuration on the port, i.e., is it part of a port group/ Link Aggregation Control Protocol (LACP) group or does it just have a VLAN assigned?

Finally are there any Media Access Control (MAC) address entries on the port?

Storage

Create an Description record to the Network Knowledge Graph against the Interface entry.

Additional Data Required?

• CRM/ CMDB circuit list.

Parameters

Apply the necessary business logic and correspond that with the description in the Network Knowledge Graph to determine the interfaces usage.

• APxx usually means access-point (wireless) and only needs a layer 2 interface back to the default gateway.

• CUSTxx usually a end-user/ subscriber facing port.

Conclusion

It is a layer 2 interface, and therefore most likely not a P2P. It will be part of a wider switching domain and as such needs to have all interfaces associated with the switching domain correctly identified, recorded and associated with a switching domain record.

Protocols

So, you want to to use protocols to discover a network, then choose wisely:

Link Level Discovery Protocol (LLDP)

One of the most useful protocols, however it is a security vulnerability vector, so a lot of operators turn it off completely.

If switched on for internal links (i.e., links within an operators Autonomous System (AS)), it can help you establish so very useful information about the devices directly connected to the element being investigated.

Example: Juniper

cr01.lab> show lldp neighbors
Local Interface	Parent Interface	Chassis Id		Port
fxp0 		- 			64:9d:99:aa:aa:aa 	Gig0/1
xe-0/1/7 	- 			64:9d:99:bb:bb:bb 	TGi0/1
et-0/0/1 	- 			64:9d:99:cc:cc:cc 	HundredGigabitEth0/1

Example: FS Switch

cs01.lab#show lldp neighbors
Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
System Name	Local Intf 	Port ID
vmh01.lab	Te0/5 		vmnic5
Unknown 	Te0/6 		b47a.f1aa.aaaa
Unknown 	Te0/8 		ecb1.d7bb.bbbb
Unknown 	Te0/9 		ecb1.d7cc.cccc
Unknown		Te0/10 		ecb1.d7dd.dddd
Unknown 	Te0/45 		d067.26ee.eeee
Unknown 	Te0/46 		PCI-E Slot 2, Port1
Unknown 	Te0/47 		PCI-E Slot 2, Port2
cs02.lab 	Hu0/49 		HundredGigabitEth0/1
cs02.lab 	Hu0/50 		HundredGigabitEth0/2
cs03.lab 	Hu0/51 		HundredGigabitEth0/3
cs04.lab 	Hu0/52 		HundredGigabitEth0/4
cr01.lab 	Hu0/53 		et-0/0/1
br01.lab 	Hu0/54 		1/1/c2/1
os01.lab 	Mg0 		Gig0/2

ISIS

Without getting deep into the ISIS vs. OSPF debate, ISIS is a powerful Interior Gateway Protocol (IGP) and has many fields associated with it to aid in understanding what devices are connected to the element being investigated.

Example: Juniper

cr01.lab> show isis adjacency
Interface 	System		L State 	Hold (secs)
et-0/0/0.0 	cr02.lab 	2 Up 		19
et-0/0/1.89 	br01.lab 	2 Up 		24
et-0/0/1.94 	vr20.lab 	2 Up 		26

Summary

Follow the link state and ARP and you’ll discover some weird and wonderful things in your network.

Additional References

https://github.com/cpmarvin/lnetd

Mindset

The following material is to take you through the mindset of an operator, rather than just WHAT IS an operator. To do this, we will take an adventure together to explore the inner workings of an operator and what drives them.

What is Telco?

Overview

Operator: "I look up to him [Regulator] because he is upper class, but I look down on him [end-user] because he is lower class."
End-User: "I know my place.”

For too long the telco world has been dominated by the push-pull mechanics of the operator exploiting the end-user and the regulator chasing their tail and over correcting.

For the UK competition in the operator space was opened up by the regulator in 2006 when OFCOM forced British Telecom (BT) to be separated into BT and Openreach.

In the US it was the break up of AT&T in 1982.

Whether it was a desired outcome of the regulators or whether it’s an organic byproduct; as the competition in the operator space ramped up, the end-user’s voice became more prominent and its fair to say that in the last 5-10yrs a operator is more driven by a end-user’s NPS score/ trying to keep a end-user’s ‘sticky’.

The negative side of this, is a race to the bottom on pricing (especially in the residential products), and so with slim margins, using traditional methods, that doesn’t leave a lot of room for high-end end-user service.

NetCo vs. ISPCo vs. Everything Else

What are these Co’s? And why does it matter?

NetCo

This is a organisation which operates a wholesale platform. Traditionally a NetCo is a natural host, and will not terminate an end-user/ subscriber session. They own and operate a network which facilitates the connectivity for the ISPCo’s. Typically they will be a Tier 1 or Tier 2 operator.

However there is a few sub-variants of a NetCo:

NHO

A natural host operator is an organisation which treats all of its customers (ISPCo) equally and fairly. It gives no organisation preferential treatment, and does not have any investment in any retail ISP’s/ customer’s (ISPCo’s).

Example - CityFibre

Vertically Integrated

This is where a NetCo strays from the neutrality, and the NetCo will have an investment in retail ISP’s/ customer’s (ISPCo’s).

Examples - Vodafone, Zen, Sky … etc.

ISPCo

Is a retail brand which requires the services of a NetCo in order to service its end-users/ subscribers. The ISPCo will terminate the subscriber session(s). It will have Autonomous System Number (ASN), RIPE entries, DNS servers … etc.

Examples - Plusnet, EE, TalkTalk (now separated from PXC) … etc.

There is also one sub-variant of an ISPCo:

vISPCo

This is a virtual ISP which is just a white labelled brand in order for said brand/ organisation to diversity its income. The organisation will let out its brand to an ISPCo/ Integrated NetCo

Examples - Post Office Broadband, Tesco’s Broadband, John Lewis Broadband … etc.

GlassCo

Is a organisation which solely provides 'glass’/ fibre. They might operate some optical repeaters in order to cover large distances, however they operate at layer 1 (physical) only.

Example - ITS.

OpCo

Is a management/ holding company for a set of Co’s either due to a shared investor or shared interests.

Why Does This Matter?

Differed Co’s have slightly different priorities, concerns and ultimately different stakeholders to satisfy.

Example

An end-user/ subscriber purchases a service from an ISPCo (TalkTalk), whom in turn purchases a service from a NetCo (PXC), whom in turn might purchase a service from a GlassCo (Openreach or CityFibre), and that only gets the connectivity to your door, the ISPCo then needs to purchase transit and peering … etc., to connect the end-user/ subscriber to the Internet/ the end-user’s/ subscriber’s favour content. And in order to do that, they need to purchase a service from a tier1 or several tier2 operators.

So the ISPCo needs to satisfy as many end-user’s/ subscriber’s as possible from tens of thousands to tens of millions, a NetCo needs to satisfy several ISPCo’s at anyone time and offer fairness/ neutrality. A GlassCo needs to get the connectivity to as many places as it is needed, all whilst satisfying landowners, government entities, the general public (whilst laying the new infrastructure) and keeping an clear, accurate and up-to-date record of where they’ve installed the new infrastructure.

By understanding a Co’s point of view, can help you leverage your position in the chain.

Tiers … what do they really mean?

The concept of ‘tiers’ these are used to signify the level of reach/ coverage that an operator has around the globe, region/ territory and/ or theatre.

Tier 1

This is traditionally seen as the ability to provide Global coverage, however in more recent times, it was re-used to mean that an operator had total coverage within country/ region/ territory and/ or theatre.

Fundamentally tier 1 has no upstream and does not pay another network to access other networks.

Global Examples:

1. Console Connect/ PCCW Global

2. BT Global

3. Colt

4. NTT Global IP Network

5. Teila/ Arelion

6. Liberty Global

Per Country/ Region/ Territory/ Theatre Examples:

1. BT UK

2. Deutsche Telekom

3. Sprint

4. Rogers

Tier 2

Traditionally a T2 was the operator with total coverage within country/ region/ territory and/ or theatre, so there’s some cross over with T1 Per Country/ Region/ Territory/ Theatre.

However more recently a T2 has morphed into an operator with coverage in a specific sub-region/ sub-geographic area(s).

Overal a T2 needs to connect to a T1 at some point, either to cross a region/ territory and/ or theatre boundary, or get access to other networks via IP transit.

Tier2 Examples

1. KCOM

2. Virgin Media O2

3. Gigalcear

Tier3

Finally there’s the T3 operators, who traditionally connect small discrete communities and/ or amalgamate several other operators connectivity to build an over the top (OTT) solution, i.e., MSPs offering SDN … etc.

View Points

Overview

The following content is to start putting ourselves in the various ‘shoes’ of the end-user/ subscriber, operator and regulator.

In the long-term however, most of these ‘mundane’ things need to be absolved or remediated to a point that it needs the minimal amount of human interaction.

An End-User/ Subscriber’s View

Q: What problem is being shown in this image?

A: The fact that an end-user had to look at their equipment.

A subscriber never touches/ looks at their ‘service’ until there is a problem, then they make it the operator's problem and without real-time clear data, they have to rely on, it’s the LED red or green?

One of the key frustrations of end-user’s is understanding the state of their service, and if it is being negatively impacted, when the service will be restored to the ‘acceptable’ level (or at least a level where they put up with it and/ or stop complaining).

An Operators View

Each operator battles daily with churn, end-user satisfaction and the over burdening regulator to try and strike a balance to stay a float.

Churn

The new paradigms of the new commercial realities (the competition factor from), has turned into a race to the bottom; especially in the residential (B2C) space.

Therefore margins per end-user are slimmer, so you need more end-user to balance your Cost of Goods and Sale (COGS).

However a new end-user also brings additional costs to ‘acquire’ them as a customer.

End User Satisfaction

And each end-user brings with them:

1. legal rights.

2. more social media accounts than anyone team can handle.

3. a complex set of requirements and environments.

Over Burdening Regulator

Whilst most regulators are like super tankers in terms of taking their time to make decisions, in the same vain they are also almost immovable objects, unless you’re a tier 1 (at least in your territory/ region/ theatre).

This means that for more operators, when the regulator says jump, you have to ask endless clarification questions to ask ‘how high?’, and ‘what for?’ .

A Regulators View

Apologies, trying to add all of the UK operators was too much for the image

A regulator will have hundreds if not thousands of operators to deal with at anyone time, coupled with millions/ billions of a region/ theatre/ territory end-users/ subscribers. And as each operator brings it’s own:

1. inherent risk/ likely hood of either taking themselves/ their end-user base offline and/ or the wider operator ecosystem in that region/ theatre/ territory.

2. market/ region to defend and/ or poach end-users from.

Key Considerations

Overview

Remember the following points are not easily solvable, however an enabler is to empower the operator to overcome/ automate the processes which frees the operator to look at the bigger picture quicker with more accurate data.

NetCo

Technical

Architecture

• Can the architecture support:

  feeds and speeds?

  • features and capabilities?

  • product set(s)?

• How will it be monitored?

• Where will the alarms/ alerts be sent to?

• What telemetry is available?

• What does BAU operationalisation look like?

• What are the Mean Time to Failure (MTTF) on the components used?

• What is the upgrade plan?

• When is the scaling point?

Outages

• What is the allowed failure rate of the products?

• Are maintenance windows included or excluded from the metrics?

Regulatory/ Compliance

• What information must be shared?

• How will the data be stored?

• When will the data be accessed? (i.e., real-time vs. long term).

• Who will access the data?

• How will the data be audited?

Commercial

ROI

• Does the product ‘wash it’s face’?

• When will the product(s) be profitable?

Compensation

• When is compensation triggered?

  • It is due to regulatory or good business practice/ product offering?

  • Are maintenances excluded?

• What form is the compensation?

• How will this affect cashflow?

Penalties

• What are the penalties for non-compliance?

  • Financial.

  • Reputation.

• How will this impact the future trading of the company?

ISPCo

You’ll notice that an ISPCo’s considerations are pretty much the same as a NetCo’s, however with a different lense/ viewpoint (hence the lack of repetition below). As it’s the ISPCo which bares most of the risk and had to deal with the most upfront in order to gain, maintain and retain the end-user/ subscriber.

Technical

Products

• Can the NetCo products support our Product Requirements?

• What products do we need to offer to differentiate ourselves in the market and stay ‘sticky’ with the end-user’s/ subscriber’s?

Outages

• What are the SLAs of the NetCo’s products?

• How will we inform the end-user’s/ subscriber’s?

Regulatory/ Compliance

• Same.

Commercial

• Same.

Touch Points

Overview

So why is all of this important? And what on earth is this guy waffling on about?

As I said before understanding an operators mindset is about helping you understand the complexities of providing a modern, scalable and fit-for-purpose service to an evolving and ever demanding end-user’s/ subscriber’s.

The Key Changes

25years ago (in the UK at least), the consumer side of Internet was something new and fun, and we were willing to wait for webpages to load, minutes for videos to buffer and hours for files to download.

Fast forward to today, and the current generation are almost on a hair trigger to do a table flip if the connectivity to their app or stream drops a few frames (ok maybe I’m exaggerating a little bit here, however you get the gist).

An Operators Current Challenges

One of the many things that operators in general have been struggling to deal with the scale of is managing a network, and with scale comes more complexity.

Within the UK AltNet community and wider NetCo’s, balancing EMS, OSS and BSS systems can become a headache that can paralyse an operator or at least put a high enough barrier to stifle innovation and prevent/ diminish the end-user/ subscriber experience.

How Can An Operator Move Forward?

Having spoken to several of the UK based operators, one of the barriers to a utopian state (other than there’s no one system to rule them all … and in the darkness bind them) is two fold:

1. to have a clear understanding of the touch points and what that then means for their network(s).

2. to leverage the data to drive manful outcomes.

The TMForm are trying to create some standards around this, however I personally believe it can broken down into these key ares and the following high level topics:

Inputs

• What is needed to drive the network?

• Is it done via the Portal and/ or an API?

Outputs

• What should/ needs to be captured?

• How can it be standardised?

Feedback

• How will the Outputs be interpreted?

• How will the interpretations be used?

Accountability

• Who will be responsible for implementation, monitoring, management, updating … etc.? (insourcing vs. outsourcing).

• How will affect Mean Time to Repair (MTTR)? In reality for an operator its often Mean Time to Innocence (MTTI).

Summary

Remember an operator is under a high account of responsibility and accountability, and so moving to data driven outcomes and a customer centric mindset is not easy.

However more about how data is an operators king/ queen/ insert diaty of choice another day.

Paraphrasing a great advert of the 1970's, it's time to P ... P ... P ... Pick up a Pirate (P... P... P... Pick up a Penguin), well not literally. However the Pirates of the ‘golden age’ have something to teach us, even 350+ years later.

Now when anyone says pirate I must admit the imagery in my mind flashes to the likes of a cartoon Captain Hook, Captain Black Beard or One-Eyed Willy, you might think of Captain Jack, Pop-up Pirate or another seafarer inspired by the great novel Treasure Island.

However whilst the Pirates weren't the first to do any of what I'm about to describe below, they brought together several threads at once to create something new and after reading Be More Pirate I'm starting to see the appeal and how many movements since have followed similar lines (either directly or indirectly).

Ultimately unhappy with the status-quo and fed-up with being treated the way they were, the Pirates set out to create a fairer, safer and as a result more productive environment.

What Did They Do?

They fought for the rights of everyone, so that each member of their crew benefited equally, from equal pay to equal rights and with the exception of a battle or a conflict (what we now call force majeure in modern corporate tones), where the Captain or the Quartermaster took charge, everyone in the crew had a vote on how the ship was run.

To do this they formed at the edges of society, out cast by the ruling elite and villainised to try and stop others joining them.

To gain traction they employed some simple, however key tactics:

• Rebels with a Cause Or How to Draw Strength by Standing Up to the Status Quo

• Rewrite Your Rules Or How to Bend, Break and Ultimately Rewrite the Rules

• Reorganise Yourself Or How to Collaborate to Achieve Scale Rather Than Growth

• Redistribute Power Or How to Fight for Fairness and Make Enemies of Exploitation

How Does This Map to UK AltNets?

The UK AltNets have become a safe harbor (much like Republic of Pirates on the Caribbean Island of Nassau in the 1710's) for those looking to perturb the status quo. However, I'm not saying we must all suddenly don eye patches, start speaking with a pirate ‘accent’ and start causing mayhem. What I am saying is that the UK AltNet’s ‘jolly roger’ was hoisted for sometime now (>10yrs), and the world has been taking notice, and so with that momentum we need to understand the principals of being a pirate as to best exploit the situation.

For the telecoms sector specifically the ‘AltNet movement’ has gained traction because a small percentage of the community had become disenfranchised with the way large operators were dominating their sub-market or geographical region. And the only way to make change is from the outside/ fringes/ edge back towards the centre.

Now in building these AltNet's they have some rather hard commercial problems to solve:

1. how to reduce the cost-to-serve so that they stay within AltNet’s funding bracket.

2. how to convert RFS/ premises passed into live services (Intacts)

3. how do you keep the operators (NetCo/ Wholesaler/ NHO) customers/ end-subscribers (Operators/ ISPCos/ Real ISP) them sticky.

Fundamentally the high level answers to those two problems are:

1. Autonomic networking.

2. Creating a platform/ market-place to allow the AltNet's commercial network(s) to form.

   1. either for NetCo/ Wholesaler/ NHO’s to ‘horse trade’ connectivity in any given ‘not-spot’ for the customers/ end-subscribers that their Operators/ ISPCos/ Real ISP’s are trying to serve.

   2. to allow Operators/ ISPCos/ Real ISP’s to shop for the ‘best’ deal from any NetCo/ Wholesaler/ NHO serving their customers/ end-subscribers.

Now there's two approaches to this:

• Come for the tool, stay for the network.

• Come for the network, stay for the tool.

Come for the tool, stay for the network

This is focused around Element Management Systems (EMS) and Network Management Systems (NMS) and is aimed at NetCo's who already have a ‘market place’, even if it is broken or hard to use/ on-board new ISPCo’s.

For these NetCo’s their biggest problem (pain point) is managing their current network estate, and any future acquisitions which brings new vendors into their ecosystems.

So a vendor neutral approach is the only way to tackle this at scale. OpenConfig has gone a long way to help solving this kind of problem, however the vendors in that ecosystem are usually shipping routing and switching hardware, which is fine for Access/ Leaf/ PE-nodes or Aggregation/ Spine/ P-Nodes or Core/ Super-Spine’s.

However, in an AltNet network, there’s far more Optical Line Termination (OLT) Passive Optical Network (PON) equipment, and there’s limited OpenConfig support from vendors creating that equipment.

Come for the network, stay for the tool

NetCo's who already have ‘sufficient’ tooling to provision their current services1, building the platform/ market-place ‘network’ is their problem (pain point), and having an flourishing ‘network’ of ISPCos transacting on their physical network it changes their value in the investment community, as well as speeds up any acquisitions and mergers (see Jack Welch's 90 day limit for integration of teams in a take-over or merger).

The same can be said of a vertically integrated operators wanting separate into NetCo and ISPCo(s). As by platform/ market-place, it allows the newly separated entities to continue to transact.

Rebels with a Cause Or How to Draw Strength by Standing Up to the Status Quo

First it challenges the vendors monopoly on how to manage, orchestrate and ultimately run a network and moves to put the power/ control back towards the NetCo by opening the freedom to chose who to source their hardware from.

Secondly it empowers NetCos by using data driven models to stop working hard, and start working smart (if you pardon the cliche). And this drives things like lowering cost-to-serve by reducing overheads and shortening mean time to innocence (MTTI)/ mean time to resolution (MTTR) … etc., reducing/ removing truck rolls … etc.

Rewrite Your Rules Or How to Bend, Break and Ultimately Rewrite the Rules

The commercial rules of telecoms have been pretty static for a long time, if there's a problem or a requirement, then a vendor will find a solution which benefits the vendor the most whilst keeping the NetCo(s) just sticky enough that they can't use another vendor.

Whilst this has been waning in recent years, and with NetCos demanding standards based solutions, rather than complete vendor proprietary lock ins.

The commercial arm of vendors have sought to influence these standards to protect their investments and minimise their costs to develop to a new standard2.

Fortunately the likes of EANTC’s multi-vendor test reports go a long way toward bridging any gaps created by vendor commercial priorities.

Longterm this approach can be ‘bent’ in the operators favour by looking to create, adapt and evolve for the great good of the subscriber, whilst keeping in mind the cost-to-serve for the NetCo. After all, we're all a subscriber of some service and the costs bore by the NetCo and indirectly passed to us via the ISPCo.

The way to do this, is to break the traditional model and move it to a truly data driven/ data centric outcomes. Several entities in the market will claim this, however they're still working within the traditions of an outdated telco model.

And the pivot point of that break comes when said data is unleashed for all decisions within the policy and intent/ data engines, and it is uses to ‘enrich’ and empower the models, systems and policies, used and set the NetCo.

Social media realised this quite sometime ago, that data is ‘king’/ ‘queen’/ ‘$insert deity of choice$’, and within social media it is unfortunately used for commercial gain. However if that data centric approach is used for the greater good, in order to better the customer/end-user experience, then we can all finally benefit.

Reorganise Yourself Or How to Collaborate to Achieve Scale Rather Than Growth

In the current corporate world, it's hard not to form into hierarchies and work top down, rather than feed bottom up. The business pressures to deliver case these hierarchies to form, and external commercial pressures from their customers drive them to keep within the status quo and limit development and innovation.

Organising work based on the requirements of the project or task, rather than working in silos and the project being passed from team to team like a hot potato; could be a resolution to removing lag in projects (hand-over and documentation problems), conflict between ‘teams’ and distributing knowledge and improving skills across the company.

Redistribute Power Or How to Fight for Fairness and Make Enemies of Exploitation

I realise I'm preaching to the converted when I say, “big business stifles information and causes a power vacuum for those at the top to benefit” and one of their ‘reasons’ for this (there's several more, however setting aside corruption and evil for a moment) is Maslow's hierarchy of needs.

Old business theory summaries to the basics of ‘give someone a job and they will be thankful for it’, then ‘give them safety and they’ll stay', finally ‘give them a carrot to progress and they’ll be happy'.

Personally, I don't believe that theory ever held true, and if it did, it was predicated around a labour-intensive market, not a knowledge worker one.

My personal mantra on this is best summarised by ‘People → Business → Finance’.

How Can I Influence This?

Whilst this isn't a battle cry to shake everything up all at once, and there is some truth in the ‘if it ain't broke …’ cliche, however that doesn't mean there isn't room for improvement. Therefore, it's time to use tooling in new and usual ways, create tooling and frameworks as needed (especially where there isn't any) and say no to doing something because ‘that the way its always been done’.

Remember that you are empowered to ask questions, push boundaries, learn something new and influence the outcome. And if you're not getting the support to do this, then please try and work with your management and/ or raise this to a senior member of staff3.

As you'll see in my final thoughts below, for me, if I'm personally not helping the company to empower itself, then I'm not doing well at being useful.

Final Thoughts

As you may a noticed, I'm rather passionate about Telecoms and improving the ecosystem in which I've operated my professional life. How ‘Be More Pirate’ resonated with me was to create my personal manifesto.

Manifesto

• ‘Be Useful’, even if that means excuse myself from meetings where I'm not contributing anything of value and/or learning.

• ‘Empower’ everyone to move the business forward, and make sure I am not a SPOF.

• ‘Be Kind’ by try and see things from other perspectives.

Recommended Reading/Book(s)

• Be More Pirate: Or How to Take On the World and Win by Sam Conniff Allende

• The Cold Start Problem: How to Start and Scale Network Effects by Andrew Chen

• The Heart of Business: Leadership Principles for the Next Era of Capitalism by Hubert Joly

After doing some research and help from colleagues at the time, the following solution was created using Juniper SRX devices.

The example below uses a vMX to separate the networks and using routing-instances and dedicated interfaces per routing-instance up to the vSRX.

Lab Topology

Objectives

In Scope

1. Provide secure routing between NOC/ management subnet(s) and the individual customers.

2. Provide the ability to manage overlapping customer subnets using static NAT IPv4 to private IPv6.

3. Prevent customer networks from communicating with each other.

Out of Scope

1. Hardening the SRX.

2. How to resolve the issue of, the NOC/ management subnet(s) overlap with the customer subnet.

3. Explicitly preventing the NOC/management IPv4 subnet(s) communicating with the customer IPv4 subnets, even if the customer IPv4 subnets overlap.

4. Routing for an alternative/backup default gateway in the customer network(s).

5. Configuration and support of the SRX’s IDP capabilities.

6. How to configure an SRX for HA.

7. Configuring DNS ALG for NAT-PT.

Assumptions

1. The NOC/management subnet(s) will:

   1. not overlap with the customer subnet.

   2. will only need the following access to the customers subnets:

      1. SSH.

      2. SNMP.

      3. PING/ICMP (Ipv4 and IPv6).

2. The customer subnets will only need access to the NOC/management subnet(s) for:

   1. AAA services (TACACs and RDP).

   2. NTP.

   3. PING/ICMP (Ipv4 and IPv6).

3. The SRX will be the default gateway for all VRFs.

Dependencies

1. The SRX has to be in ‘flow-based’ mode for IPv4 and IPv6 and rebooted before any of the below commands are used:

   1. IPv4 - https://kb.juniper.net/InfoCenter/index?page=content&id=KB30461&cat=J_SERIES&actp=LIST

   2. IPv6 - https://kb.juniper.net/InfoCenter/index?page=content&id=KB25697&actp=METADATA

2. The IPv6 subnet cannot be smaller than the IPv4 subnet it’s being mapped to .

3. The application-identification has to be installed before any of the below commands are used:

   1. Download - https://www.juniper.net/documentation/us/en/software/junos/application-identification/topics/ref/command/request-services-application-identification-download.html.

   2. Install - https://www.juniper.net/documentation/us/en/software/junos/application-identification/topics/ref/command/request-services-application-identification-install.html.

   3. Check to see what is installed - https://www.juniper.net/documentation/us/en/software/junos/application-identification/topics/ref/command/show-services-application-identification-application.html or https://www.juniper.net/documentation/us/en/software/junos/cli/topics/ref/command/show-groups-junos-defaults.html.

So How Does It Work?

Packet Walk

Steps

1. NAT46 rule

2. For static NAT46 SRX will automatically create the reverse of step 1, but the SRX will never put it into the configuration output.

3. Now before the packet can be passed into trust-103 we need to use NAT641.

4. The reverse of step 3, but the SRX will never put it into the configuration output.

Configlets

Step 1

from zone trust-101
match destination-address 10.0.103.0/24
static-nat prefix 2001::103:0/120
static-nat prefix routing-instance DC-103

Step 2

from zone trust-103
match destination-address 2001::103:0/120 
static-nat prefix 10.0.103.0/24
static-nat prefix routing-instance DC-101

Step 3

from zone trust-103
match destination-address fd00::101:0/120
static-nat prefix 192.168.101.0/24
static-nat prefix routing-instance DC-101

Step 4

from zone trust-101
match destination-address 192.168.101.0/24
static-nat prefix fd00::101:0/120
static-nat prefix routing-instance DC-103

Sandbox Topology

Working Examples

IPv4 flows verification

Confirm the firewall rule(s) and the action for a packet type:

vSRX-DC# run show security match-policies source-ip 192.168.101.2 destination-ip 2001::103:2 source-port 1 destination-port 1 protocol icmp from-zone trust-101 to-zone trust-103   
Policy: default-permit, action-type: permit, State: enabled, Index: 6
0
  Policy Type: Configured
  Sequence number: 1
  From zone: trust-101, To zone: trust-103
  Source vrf group:
    any
  Destination vrf group:
    any
  Source addresses:
    DC-101_IPv4(global): 192.168.101.0/24
  Destination addresses:
    DC-103_IPv6(global): 2001::103:0/120
  Application: CUST-ACCESS
    IP protocol: icmp, ALG: 0, Inactivity timeout: 60
      ICMP Information: type=255, code=0
    IP protocol: 58, ALG: 0, Inactivity timeout: 60
      ICMP Information: type=255, code=0
    IP protocol: tcp, ALG: 0, Inactivity timeout: 1800
      Source port range: [0-0] 
      Destination ports: 3389
    IP protocol: tcp, ALG: 0, Inactivity timeout: 1800
      Source port range: [0-0] 
      Destination ports: 49
    IP protocol: tcp, ALG: 0, Inactivity timeout: 1800
      Source port range: [0-0] 
      Destination ports: 65
  Per policy TCP Options: SYN check: No, SEQ check: No, Window scale: No

Host to Host Communication Verification

DC-101 to DC-103

Ping from SERVER-1 in DC-101 to SERVER-3 in DC-103

ubuntu:~$ ping 192.168.103.2 -c 100
PING 192.168.103.2 (192.168.103.2) 56(84) bytes of data.
64 bytes from 192.168.103.2: icmp_seq=1 ttl=61 time=2.45 ms
64 bytes from 192.168.103.2: icmp_seq=1 ttl=61 time=2.45 ms
64 bytes from 192.168.103.2: icmp_seq=1 ttl=61 time=2.45 ms
.... <SNIP>

--- 192.168.103.2 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.455/2.455/2.455/0.000 ms

Verify the session flow creation to static NAT address2:

vSRX-HA> show security flow session source-prefix 192.168.101.0/24 extensive    
Session ID: 12607, Status: Normal
Flags: 0x4000000/0x0/0x2/0x2000003
Policy name: default-permit/6
Source NAT pool: Null
Dynamic application: junos:UNKNOWN, 
Encryption:  Unknown
Url-category:  Unknown
Application traffic control rule-set: INVALID, Rule: INVALID
Maximum timeout: 60, Current timeout: 2
Session State: Valid
Start time: 165580, Duration: 3
   In: 192.168.101.2/17189 --> 10.0.103.2/15;icmp, 
  Conn Tag: 0x0, Interface: ge-0/0/4.0, 
    Session token: 0x700a, Flag: 0x621
    Route: 0xf0010, Gateway: 172.16.100.1, Tunnel ID: 0, Tunnel type: None
    Port sequence: 0, FIN sequence: 0, 
    FIN state: 0, 
    Pkts: 1, Bytes: 84 
   Out: 2001::103:2/15 --> fd00::101:2/17189;icmp6, 
  Conn Tag: 0x0, Interface: ge-0/0/7.0, 
    Session token: 0x900c, Flag: 0x622
    Route: 0x130010, Gateway: 2001::100:1, Tunnel ID: 0, Tunnel type: None
    Port sequence: 0, FIN sequence: 0, 
    FIN state: 0, 
    Pkts: 1, Bytes: 104
Total sessions: 1

Note: Check the byte size on the return session to make sure they match in the return direction.

How SERVER-3 in DC-103 sees the ping request from SERVER-1 in DC-101:

ubuntu:~$ sudo tcpdump -i eth3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 262144 bytes
06:51:37.868620 IP6 fd00::101:2 > 2001::103:2: ICMP6, echo request, seq 1, length 64
06:51:37.868673 IP6 2001::103:2 > fd00::101:2: ICMP6, echo reply, seq 1, length 64

DC-103 to DC-101

Ping from SERVER-3 in DC-103 to SERVER-1 in DC-101:

root@ubuntu:~# ping fd00::101:2 -c100
PING fd00::101:2(fd00::101:2) 56 data bytes
64 bytes from fd00::101:2: icmp_seq=1 ttl=61 time=2.67 ms
64 bytes from fd00::101:2: icmp_seq=1 ttl=61 time=2.67 ms
64 bytes from fd00::101:2: icmp_seq=1 ttl=61 time=2.67 ms
... <SNIP>
--- fd00::101:2 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.613/2.613/2.613/0.000 ms

Verify the session flow creation to static NAT address3:

vSRX-DC> show security flow session source-prefix 2001::103:0/120 extensive    
Session ID: 12757, Status: Normal
Flags: 0x84000000/0x0/0x2/0x2800003
Policy name: default-permit/12
Source NAT pool: Null
Dynamic application: junos:UNKNOWN, 
Encryption:  Unknown
Url-category:  Unknown
Application traffic control rule-set: INVALID, Rule: INVALID
Maximum timeout: 4, Current timeout: 2
Session State: Valid
Start time: 168194, Duration: 2
   In: 2001::103:2/17083 --> fd00::101:2/1;icmp6, 
  Conn Tag: 0x0, Interface: ge-0/0/7.0, 
    Session token: 0x900c, Flag: 0x623
    Route: 0x130010, Gateway: 2001::100:1, Tunnel ID: 0, Tunnel type: None
    Port sequence: 0, FIN sequence: 0, 
    FIN state: 0, 
    Pkts: 1, Bytes: 104
   Out: 192.168.101.2/1 --> 10.0.103.2/17083;icmp, 
  Conn Tag: 0x0, Interface: ge-0/0/4.0, 
    Session token: 0x700a, Flag: 0x620
    Route: 0xf0010, Gateway: 172.16.100.1, Tunnel ID: 0, Tunnel type: None
    Port sequence: 0, FIN sequence: 0, 
    FIN state: 0, 
    Pkts: 1, Bytes: 84
Total sessions: 1

How SERVER-1 in DC-101 sees the ping request from SERVER-3 in DC-103

root@ubuntu:~# tcpdump -i eth1
[sudo] password for jcluser: 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes

07:33:47.126039 IP 10.0.103.2 > 192.168.101.2: ICMP echo request, id 17080, seq 1, length 64
07:33:47.126092 IP 192.168.101.2 > 10.0.103.2: ICMP echo reply, id 17080, seq 1, length 64

Sandbox Preparation

Linux Hosts Configuration

Configure each Ubuntu server IP address and static route:

# Server-1
network:
    version: 2
    renderer: networkd
    ethernets:
        eth1:
         addresses:
             - 192.168.101.2/24
         routes:
             - to: 192.168.0.0/16
               via: 192.168.101.1
             - to: 172.16.0.0/12
               via: 192.168.101.1
# Server-2
network:
    version: 2
    renderer: networkd
    ethernets:
        eth2:
         addresses:
             - 192.168.101.2/24
         routes:
             - to: 192.168.0.0/16
               via: 192.168.101.1
             - to: 172.16.0.0/12
               via: 192.168.101.1
# Server-3
network:
    version: 2
    renderer: networkd
    ethernets:
        eth3:
         addresses:
             - 192.168.103.2/24
             - 2001::103:2/120
         routes:
             - to: 192.168.0.0/16
               via: 192.168.103.1
             - to: 10.0.0.0/8
               via: 192.168.103.1
             - to: 172.16.0.0/12
               via: 192.168.103.1
             - to: 2001::100:0/122
               via: 2001::103:1
             - to: fd00::/8
               via: 2001::103:1
# Server-4
network:
    version: 2
    renderer: networkd
    ethernets:
        eth4:
         addresses:
             - 2001::104:2/120
         routes:
             - to: 2001::100:0/122
               via: 2001::104:1
             - to: fd00::/8
               via: 2001::104:1

Configure each Ubuntu servers to access the ‘Internet’ via the vSRX with:

# Internet-Server
nmcli -p dev
ifconfig eth1
sudo ip route add 172.16.0.0/12 via 172.30.30.1
sudo vi /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE="eth1"
ONBOOT=yes
BOOTPROTO=static
IPADDR=172.30.30.2
NETMASK=255.255.255.252

systemctl restart network

vMX and vSRX Configuration

vMX

vMX Ifd Configuration

Create the interfaces between the vMX and Linux hosts, and the vMX and the vSRX:

set interfaces ge-0/0/0 description SRX_ge-0/0/0_DC-101
set interfaces ge-0/0/0 unit 0 family inet address 172.16.100.1/30

set interfaces ge-0/0/1 description SRX_ge-0/0/5_DC-102
set interfaces ge-0/0/1 unit 0 family inet address 172.16.100.5/30 

set interfaces ge-0/0/2 description SRX_ge-0/0/6_DC-103
set interfaces ge-0/0/2 unit 0 family inet address 172.16.100.9/30 
set interfaces ge-0/0/2 unit 0 family inet6 address 2001::100:1/125 

set interfaces ge-0/0/3 description SRX_ge-0/0/7_DC-104
set interfaces ge-0/0/3 unit 0 family inet address 172.16.100.13/30 
set interfaces ge-0/0/3 unit 0 family inet6 address 2001::100:13/125
 
set interfaces ge-0/0/4 description SERVER1
set interfaces ge-0/0/4 unit 0 family inet address 192.168.101.1/24 

set interfaces ge-0/0/5 description SERVER2
set interfaces ge-0/0/5 unit 0 family inet address 192.168.101.1/24
 
set interfaces ge-0/0/6 description SERVER3
set interfaces ge-0/0/6 unit 0 family inet address 10.0.103.1/24 
set interfaces ge-0/0/6 unit 0 family inet6 address 2001::103:1/120

set interfaces ge-0/0/7 description SERVER4
set interfaces ge-0/0/7 unit 0 family inet address 192.168.104.1/24 
set interfaces ge-0/0/7 unit 0 family inet6 address 2001::104:1/120

vMX Routing-Instance VRF Configuration

Create a Routing-Instance VRF and create a BGP session per VRF towards vSRX4:

set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 type internal
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 local-address 172.16.100.1
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 family inet unicast
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 export DIRECT
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 neighbor 172.16.100.2
set routing-instances DC-101 instance-type vrf
set routing-instances DC-101 interface ge-0/0/0.0
set routing-instances DC-101 interface ge-0/0/4.0
set routing-instances DC-101 route-distinguisher 100:65001
set routing-instances DC-101 vrf-target target:100:100
set routing-instances DC-101 vrf-table-label

set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 type internal
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 local-address 172.16.100.5
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 family inet unicast
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 export DIRECT
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 neighbor 172.16.100.6
set routing-instances DC-102 instance-type vrf
set routing-instances DC-102 interface ge-0/0/1.0
set routing-instances DC-102 interface ge-0/0/5.0
set routing-instances DC-102 route-distinguisher 200:65001
set routing-instances DC-102 vrf-target target:200:200
set routing-instances DC-102 vrf-table-label

set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 type internal
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 local-address 172.16.100.9
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 family inet unicast
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 export DIRECT
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 neighbor 172.16.100.10
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 type internal
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 local-address 2001::100:1
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 family inet6 unicast
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 export DIRECT-INET6
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 neighbor 2001::100:2
set routing-instances DC-103 instance-type vrf
set routing-instances DC-103 interface ge-0/0/2.0
set routing-instances DC-103 interface ge-0/0/6.0
set routing-instances DC-103 route-distinguisher 300:65001 
set routing-instances DC-103 vrf-target target:300:300 
set routing-instances DC-103 vrf-table-label

set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 type internal
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 local-address 172.16.100.13
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 family inet unicast
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 export DIRECT
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 neighbor 172.16.100.14
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 type internal
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 local-address 2001::100:13
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 family inet6 unicast
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 export DIRECT-INET6
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 neighbor 2001::100:14
set routing-instances DC-104 instance-type vrf
set routing-instances DC-104 interface ge-0/0/3.0
set routing-instances DC-104 interface ge-0/0/7.0
set routing-instances DC-104 route-distinguisher 400:65001 
set routing-instances DC-104 vrf-target target:400:400 
set routing-instances DC-104 vrf-table-label

set routing-options autonomous-system 65535

vMX Route Filters

Advertise direct addresses only with a route-filter:

set policy-options policy-statement DIRECT term DIRECT from protocol direct
set policy-options policy-statement DIRECT term DIRECT from route-filter 192.168.101.0/24 exact
set policy-options policy-statement DIRECT term DIRECT from route-filter 192.168.103.0/24 exact
set policy-options policy-statement DIRECT term DIRECT then accept

set policy-options policy-statement DIRECT-INET6 term DIRECT from protocol direct
set policy-options policy-statement DIRECT-INET6 term DIRECT from route-filter 2001::103:0/120 exact
set policy-options policy-statement DIRECT-INET6 term DIRECT from route-filter 2001::104:0/120 exact
set policy-options policy-statement DIRECT-INET6 term DIRECT then accept

vSRX

vSRX Ifd Configuration

Create the interfaces between the vSRX and the vMX:

set interfaces ge-0/0/0 description TO-DC-101
set interfaces ge-0/0/0 unit 0 family inet address 172.16.100.2/30

set interfaces ge-0/0/1 description TO-DC-102
set interfaces ge-0/0/1 unit 0 family inet address 172.16.100.6/30

set interfaces ge-0/0/2 description TO-DC-103
set interfaces ge-0/0/2 unit 0 family inet address 172.16.100.10/30
set interfaces ge-0/0/2 unit 0 family inet6 address 2001::100:2/125

set interfaces ge-0/0/3 description TO-DC-104
set interfaces ge-0/0/3 unit 0 family inet address 172.16.100.14/30
set interfaces ge-0/0/3 unit 0 family inet6 address 2001::100:14/125

set interfaces ge-0/0/4 description TO-INTERNET
set interfaces ge-0/0/4 unit 0 family inet address 172.30.30.1/30

vSRX Routing-Instance VRF Configuration

Routing-instance VR configuration and BGP session towards vMX. Each VR with default route which serves 2 purposes:

1. Advertise default route to vMX.

2. Route leak to default for internet traffic5.

set routing-instances DC-101 routing-options static route 0.0.0.0/0 next-table inet.0
set routing-instances DC-101 routing-options static route 10.0.103.0/24 next-table DC-103.inet.0
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 type internal
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 local-address 172.16.100.2
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 family inet unicast
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 export PER-VR-ADV
set routing-instances DC-101 protocols bgp group ibgp-ipv4-101 neighbor 172.16.100.1
set routing-instances DC-101 interface ge-0/0/0.0
set routing-instances DC-101 instance-type virtual-router

set routing-instances DC-102 routing-options static route 10.0.103.0/24 next-table DC-103.inet.0
set routing-instances DC-102 routing-options static route 0.0.0.0/0 next-table inet.0
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 type internal
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 local-address 172.16.100.6
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 family inet unicast
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 export PER-VR-ADV
set routing-instances DC-102 protocols bgp group ibgp-ipv4-102 neighbor 172.16.100.5
set routing-instances DC-102 interface ge-0/0/1.0
set routing-instances DC-102 instance-type virtual-router

set routing-instances DC-103 routing-options rib DC-103.inet6.0 static route ::/0 discard
set routing-instances DC-103 routing-options rib DC-103.inet6.0 static route 2001::104:0/122 next-table DC-104.inet6.0
set routing-instances DC-103 routing-options static route 0.0.0.0/0 next-table inet.0
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 type internal
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 local-address 172.16.100.10
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 family inet unicast
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 export PER-VR-ADV
set routing-instances DC-103 protocols bgp group ibgp-ipv4-103 neighbor 172.16.100.9
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 type internal
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 local-address 2001::100:2
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 family inet6 unicast
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 export PER-VR-ADV-INET6
set routing-instances DC-103 protocols bgp group ibgp-ipv6-101 neighbor 2001::100:1
set routing-instances DC-103 interface ge-0/0/2.0
set routing-instances DC-103 instance-type virtual-router

set routing-instances DC-104 routing-options rib DC-104.inet6.0 static route ::/0 discard
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 type internal
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 local-address 172.16.100.14
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 family inet unicast
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 export PER-VR-ADV
set routing-instances DC-104 protocols bgp group ibgp-ipv4-104 neighbor 172.16.100.13
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 type internal
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 local-address 2001::100:14
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 family inet6 unicast
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 export PER-VR-ADV-INET6
set routing-instances DC-104 protocols bgp group ibgp-ipv6-104 neighbor 2001::100:13
set routing-instances DC-104 interface ge-0/0/3.0
set routing-instances DC-104 instance-type virtual-router

set routing-options autonomous-system 65535

vSRX VRF to VRF Access

vSRX Security Zones

Create the security zones with:

set security zones security-zone untrust screen untrust-screen
set security zones security-zone untrust host-inbound-traffic protocols bgp
set security zones security-zone untrust interfaces ge-0/0/4.0

set security zones security-zone trust-101 tcp-rst
set security zones security-zone trust-101 host-inbound-traffic protocols bgp
set security zones security-zone trust-101 host-inbound-traffic system-services ping
set security zones security-zone trust-101 interfaces ge-0/0/0.0

set security zones security-zone trust-102 tcp-rst
set security zones security-zone trust-102 host-inbound-traffic protocols bgp
set security zones security-zone trust-102 host-inbound-traffic system-services ping
set security zones security-zone trust-102 interfaces ge-0/0/1.0

set security zones security-zone trust-103 tcp-rst
set security zones security-zone trust-103 host-inbound-traffic protocols bgp
set security zones security-zone trust-103 host-inbound-traffic system-services ping
set security zones security-zone trust-103 interfaces ge-0/0/2.0

set security zones security-zone trust-104 tcp-rst
set security zones security-zone trust-104 host-inbound-traffic protocols bgp
set security zones security-zone trust-104 host-inbound-traffic system-services ping
set security zones security-zone trust-104 interfaces ge-0/0/3.0

vSRX ‘Internet’ Access Security Policies

Allow the VRFs access to the ‘Internet’ with:

set security policies from-zone trust-101 to-zone untrust policy default-permit match source-address any
set security policies from-zone trust-101 to-zone untrust policy default-permit match destination-address any
set security policies from-zone trust-101 to-zone untrust policy default-permit match application any
set security policies from-zone trust-101 to-zone untrust policy default-permit then permit

set security policies from-zone trust-102 to-zone untrust policy default-permit match source-address any
set security policies from-zone trust-102 to-zone untrust policy default-permit match destination-address any
set security policies from-zone trust-102 to-zone untrust policy default-permit match application any
set security policies from-zone trust-102 to-zone untrust policy default-permit then permit

set security policies from-zone trust-103 to-zone untrust policy default-permit match source-address any
set security policies from-zone trust-103 to-zone untrust policy default-permit match destination-address any
set security policies from-zone trust-103 to-zone untrust policy default-permit match application any
set security policies from-zone trust-103 to-zone untrust policy default-permit then permit

set security policies from-zone trust-104 to-zone untrust policy default-permit match source-address any
set security policies from-zone trust-104 to-zone untrust policy default-permit match destination-address any
set security policies from-zone trust-104 to-zone untrust policy default-permit match application any
set security policies from-zone trust-104 to-zone untrust policy default-permit then permit

vSRX MGMT Access

Create security policies between MGMT VR and Customer VRs6 with:

set security address-book global address DC-101_IPv4 192.168.101.0/24
set security address-book global address DC-102_IPv4 192.168.101.0/24
set security address-book global address DC-103_IPv4 10.0.103.0/24
set security address-book global address DC-103_IPv6 2001::103:0/120
set security address-book global address DC-104_IPv6 2001::104:0/120

set security policies from-zone trust-101 to-zone trust-103 policy default-permit match source-address DC-101_IPv4
set security policies from-zone trust-101 to-zone trust-103 policy default-permit match destination-address DC-103_IPv6
set security policies from-zone trust-101 to-zone trust-103 policy default-permit match application CUST-ACCESS
set security policies from-zone trust-101 to-zone trust-103 policy default-permit then permit
set security policies from-zone trust-101 to-zone trust-104 policy default-permit match source-address DC-101_IPv4
set security policies from-zone trust-101 to-zone trust-104 policy default-permit match destination-address DC-104_IPv6
set security policies from-zone trust-101 to-zone trust-104 policy default-permit match application CUST-ACCESS
set security policies from-zone trust-101 to-zone trust-104 policy default-permit then permit

set security policies from-zone trust-102 to-zone trust-103 policy default-permit match source-address DC-102_IPv4
set security policies from-zone trust-102 to-zone trust-103 policy default-permit match destination-address DC-103_IPv6
set security policies from-zone trust-102 to-zone trust-103 policy default-permit match application CUST-ACCESS
set security policies from-zone trust-102 to-zone trust-103 policy default-permit then permit
set security policies from-zone trust-102 to-zone trust-104 policy default-permit match source-address DC-102_IPv4
set security policies from-zone trust-102 to-zone trust-104 policy default-permit match destination-address DC-104_IPv6
set security policies from-zone trust-102 to-zone trust-104 policy default-permit match application CUST-ACCESS
set security policies from-zone trust-102 to-zone trust-104 policy default-permit then permit

set security policies from-zone trust-103 to-zone trust-101 policy default-permit match source-address DC-103_IPv6
set security policies from-zone trust-103 to-zone trust-101 policy default-permit match destination-address DC-101_IPv4
set security policies from-zone trust-103 to-zone trust-101 policy default-permit match application NOC-MGMT
set security policies from-zone trust-103 to-zone trust-101 policy default-permit then permit
set security policies from-zone trust-103 to-zone trust-102 policy default-permit match source-address DC-103_IPv6
set security policies from-zone trust-103 to-zone trust-102 policy default-permit match destination-address DC-102_IPv4
set security policies from-zone trust-103 to-zone trust-102 policy default-permit match application NOC-MGMT
set security policies from-zone trust-103 to-zone trust-102 policy default-permit then permit

set security policies from-zone trust-104 to-zone trust-101 policy default-permit match source-address DC-104_IPv6
set security policies from-zone trust-104 to-zone trust-101 policy default-permit match destination-address DC-101_IPv4
set security policies from-zone trust-104 to-zone trust-101 policy default-permit match application NOC-MGMT
set security policies from-zone trust-104 to-zone trust-101 policy default-permit then permit
set security policies from-zone trust-104 to-zone trust-102 policy default-permit match source-address DC-104_IPv6
set security policies from-zone trust-104 to-zone trust-102 policy default-permit match destination-address DC-102_IPv4
set security policies from-zone trust-104 to-zone trust-102 policy default-permit match application NOC-MGMT
set security policies from-zone trust-104 to-zone trust-102 policy default-permit then permit

vSRX Source NAT

DC-101 and DC-102 to Internet

set security nat source rule-set DC-101-INTERNET-RS from zone trust-101
set security nat source rule-set DC-101-INTERNET-RS to zone untrust
set security nat source rule-set DC-101-INTERNET-RS rule internet-rule-1 match source-address 192.168.101.0/24
set security nat source rule-set DC-101-INTERNET-RS rule internet-rule-1 match destination-address 0.0.0.0/0
set security nat source rule-set DC-101-INTERNET-RS rule internet-rule-1 then source-nat interface

set security nat source rule-set DC-102-INTERNET-RS from zone trust-102
set security nat source rule-set DC-102-INTERNET-RS to zone untrust
set security nat source rule-set DC-102-INTERNET-RS rule internet-rule-1 match source-address 192.168.101.0/24
set security nat source rule-set DC-102-INTERNET-RS rule internet-rule-1 match destination-address 0.0.0.0/0
set security nat source rule-set DC-102-INTERNET-RS rule internet-rule-1 then source-nat interface

DC-103 pool to Internet

It is assumed that DC-104 doesn’t need access to the internet.

set security nat source pool DC-103-PUBLIC address 172.30.103.0/24
set security nat source rule-set DC-103-INTERNET-RS from zone trust-103
set security nat source rule-set DC-103-INTERNET-RS to zone untrust
set security nat source rule-set DC-103-INTERNET-RS rule internet-rule-2 match source-address 192.168.103.0/24
set security nat source rule-set DC-103-INTERNET-RS rule internet-rule-2 match destination-address 0.0.0.0/0
set security nat source rule-set DC-103-INTERNET-RS rule internet-rule-2 then source-nat pool DC-103-PUBLIC

vSRX Static IPv4 (NAT46) and IPv6 (NAT64) NAT

set security nat static rule-set NAT46 from zone [ trust-101 trust-102 ]
set security nat static rule-set NAT46 rule rule-1 match destination-address 10.0.103.0/24
set security nat static rule-set NAT46 rule rule-1 then static-nat prefix 2001::103:0/120
set security nat static rule-set NAT46 rule rule-1 then static-nat prefix routing-instance DC-103
set security nat static rule-set NAT46 rule rule-2 match destination-address 10.0.104.0/24
set security nat static rule-set NAT46 rule rule-2 then static-nat prefix 2001::104:0/120
set security nat static rule-set NAT46 rule rule-2 then static-nat prefix routing-instance DC-104

set security nat static rule-set NAT64 from zone [ trust-103 trust-104 ]
set security nat static rule-set NAT64 rule rule-1 match destination-address fd00::101:0/120
set security nat static rule-set NAT64 rule rule-1 then static-nat prefix 192.168.101.0/24
set security nat static rule-set NAT64 rule rule-1 then static-nat prefix routing-instance DC-101
set security nat static rule-set NAT64 rule rule-2 match destination-address fd00::102:0/120
set security nat static rule-set NAT64 rule rule-2 then static-nat prefix 192.168.101.0/24
set security nat static rule-set NAT64 rule rule-2 then static-nat prefix routing-instance DC-102

Appendices

Address Book and Address-Set

To keep the policies light address-set’s can be used to group several address book entries together - https://www.juniper.net/documentation/us/en/software/junos/security-policies/topics/topic-map/security-address-books-sets.html.

High Availability for SRX

1. SRX HA Configuration Generator - https://support.juniper.net/support/tools/srxha/.

2. Configuring Chassis Clustering on SRX Series Devices - https://www.juniper.net/documentation/us/en/software/junos/chassis-cluster-security-devices/topics/topic-map/security-chassis-cluster-verification.html.

Intrusion Detection and Prevention (IDP) Links

1. How to configure IDP on the SRX Series -

a. SRX Getting Started - Install license for Antivirus, Web Filter, IDP, or Antispam - http://kb.juniper.net/KB16675.

2. Juniper vSRX: Advanced Security Feature Demo -

3. Juniper SRX NGFW Walkthrough -

SRX Pre-Defined Applications and Address Books

1. Address Book - https://www.juniper.net/documentation/us/en/software/junos/security-policies/topics/topic-map/security-address-books-sets.html.

2. Pre-defined applications - https://www.juniper.net/documentation/us/en/software/junos/security-policies/topics/topic-map/policy-predefined-applications.html, (also see Dependencies).

Traffic Processing on SRX Series

Traffic Processing on SRX Series Devices Overview - https://www.juniper.net/documentation/us/en/software/junos/flow-packet-processing/topics/topic-map/security-srx-devices-processing-overview.html.

SRX Static NAT46 <=> NAT64, why we need both and they’re part of NAT-PT Overview

There are several Juniper documents covering this topic, but they don’t quite cover the use case of this PoC. The requirement of the PoC is to translate any IPv4 address to an IPv6 address, but keep the last octet the same, i.e., 192.168.101.2 becomes fd00::101:2, therefore a static NAT will be used.

As the SRX is stateful, static NATs automatically create the reverse rule in the SRX, but never display it in the configuration. That allows for a more condensed rule base in the SRX. However, the need for NAT-PT is that, as the IP address translated using NAT46 only the destination IP has been translated IPv4 to IPv6, therefore at thew moment (without any further processing), we have a packet with an IPv4 source and an IPv6 destination and therefore routing will not work. Therefore, we need a NAT64 rule to complete the source IP translation; hence NAT-PT.

Example

The ‘forward’ NAT46 rule from the NOC Management zone/VR to the customer zone/VR:

set security nat static rule-set NAT46 from zone trust-101
set security nat static rule-set NAT46 rule rule-1 match destination-address 10.0.103.0/24
set security nat static rule-set NAT46 rule rule-1 then static-nat prefix 2001::103:0/120
set security nat static rule-set NAT46 rule rule-1 then static-nat prefix routing-instance DC-103

From this ‘forward’ NAT46 rule, we map anything that is originated in DC-103 VR (zone trust-103) from the IP subnet 2001::103:0/120, NAT it to IP subnet 10.0.103.0/24, and put the NAT’d traffic into zone trust-101 (DC-101 VR).

The ‘reverse’ NAT64 rule from the customer zone/ VR to the NOC Management zone/ VR:

set security nat static rule-set NAT64 from zone trust-103
set security nat static rule-set NAT64 rule rule-1 match destination-address fd00::101:0/120
set security nat static rule-set NAT64 rule rule-1 then static-nat prefix 192.168.101.0/24
set security nat static rule-set NAT64 rule rule-1 then static-nat prefix routing-instance DC-101

From this ‘reverse’ NAT64 rule, we map anything that is originated in DC-101 VR (zone trust-101) from the IP subnet 192.168.101.0/24, NAT it to IP subnet fd00::101:0/120, and put the NAT’d traffic into zone trust-103 (DC-103 VR).

Links

1. Configuring NAT46 on SRX devices - https://kb.juniper.net/InfoCenter/index?page=content&id=KB33559&actp=METADATA.

2. IPv6 NAT - https://www.juniper.net/documentation/us/en/software/junos/nat/topics/topic-map/security-ipv6-nat.html#d113e35.

Intro

Hot on the heels of Mobile World Congress (MWC) comes MPLS WC. Now in its 25th year, whilst the conference hit a key milestone, it didn't bring anything new to the table.

The event has two main tracks, the first focusing on MPLS/ Service Provider and the second on Quantum with AI & ML.

This series of post will focus on my feedback on the MPLS/ Service Provider highlights.

Looking forward to the Future of the Internet Protocol

The conference kicked off with a flurry of conversations around:

• how to leverage IPv6 to deliver application/content aware networking, for both customer experience

• to ensure ‘security’ (trust) on a given network path as well as keeping a subscribers data within the data region required by the governing law.

• building Network Digital Twins (NDT) to test new features, to help move towards automation and playback failure scenarios to investigate route causes … etc.

Unfortunately for me the conversation from most vendors then started to 'turn south', as they all began to display that they've been drinking the 'SRv6 Kool-Aid'.

Cisco

Want to reinvent another way to monitor the existing services, for no good reason by using SRv6.

Juniper

Pushing the NDT as a key concept for:

Ciena

Realising that there's money in data:

Nokia

Came out swinging, showing their support (founding) of a ‘new’ open-source automation tool:

AvidThink

“it's an API economy”

So, what does this all mean anyways?

The industry catching up with the idea of applying business rules and logic to enriched data using ML, and that autonomic networking is the future … even if they think it will take another 20yrs.

Integrated Performance Measurement Session

Whilst my tolerance levels for hearing about SRv6 were severally tested, there was some light around Internetwork Performance Monitor (IPM) and Cisco summed it up the best:

And Colt is waking up to the idea that powering networks cost money, and insinuated that they would be looking to route their customers services based on cost, as long as it met the agreed SLA/KPIs:

And finally, I was disappointed when Verizon introduced KISS and didn't mean the classic rock band.

SRv6

What is SRv6?

SRv6 is an evolution in networking protocols which will have some significant benefits, however it's isn't fully finished yet in the various standards committees and as such isn't ready to deliver the potential benefits, nor should it be considered stable for a production environment.

What was my feedback to the Vendors?

I used the following analogy:

“It's like the movie Elf when they realise the book is missing two pages, however at the moment all the vendors are signing the pages as if SRv6 is ready to go, and it's not. There are several parts to complete, so why would anyone roll it out into a production network unless they could back it off to a vendor?”

Simon Beevers

So, that only leaves the Tier 1 ISPs who are able to deploy this with the direct backing of a vendor, and whilst this will create useful production data, as a subscriber to an ISP service, I don't want my connectivity built over ‘an unfinished story’.

Alternative

SR-MPLS using IPv6 native networking, (SR-MPLSv6 or SRo6 as I prefer to call it 😉), however there is limited support for this as most vendors need to program the IPv6 node SIDs for SR-MPLS.

KISS

Keep it Simple & Strategic

Everyone in an organisation, needs to feel empowered not only complete the day-to-day tasks and assignments, but to flag anything they see whether that be an improvement, a fault or a potential future problem; both within their direct domain or outside of it.

So Why is This so Important?

As a leader I can say, I'd rather spend 10-15mins having a conversation with someone who's flagging something to me, so that I can empower them to spend time pursuing the opportunity, or start assessing the risks and impact (if any) of a potential problem.

Also no-one has the bandwidth or capability to be involved with all projects at all times and therefore be aware of everything that's going on. So as a leader I rely on my team and those around me to subject matter experts as well as to be my eyes and ears when I can't be there, and finally bring domain knowledge and background to a situation.

A Short Story

To emphasise my point, here's an example which points out that organisations can be “organised to neglect their awareness. Like most companies, Expedia divided its workforce into groups, each with its own focus.”.

This in my opinion, needs to be avoided at all costs, as this neglect of awareness leads to inefficiencies, disorganisation, blame culture and the dreaded “not my problem”.

References

Recommended Reading/Book(s)

• Upstream: How to solve problems before they happen Paperback by Dan Heath

Merchant Silicon (AKA Whitebox) is equipment created independent vendors which can run any Network Operating System (NOS) programmed for it, rather than proprietary hardware and a proprietary NOS.

What isn’t Merchant Silicon?

Any vendor (’traditional vendor’) that is vertically integrated, usually using proprietary hardware and their own proprietary NOS.

What is a NOS?

A NOS is much like a OS on a computer. It takes intent from the user and via policy programs the hardware. NOS’s overall are CLI driven, rather than having a nice graphical UI, so network operators have to learn to love the DOS like CLI, however some NOS’s do come with a graphical UI to obfuscate the CLI commands required to program the hardware.

These UIs (typically firewalls or enterprise devices) bring with them their own complexities such as, creating bloat in the naming conventions, completely hiding some essential commands from the CLI, and/ or being an attack vector for threat actors.

Who provides this Merchant Silicon Hardware?

Two of the main vendors of Merchant Silicon are EdgeCore and ufiSpace, however there are several others and some who will build custom hardware as needed.

Merchant Silicon Analogy - PC vs. Mac

A ‘PC’ can be built and configured using any compatible set of hardware and the operating system (OS) can either be a fully supported OS (Windows) or a community one (Linux) or one of your own creation.

A Mac on the other hand is completely proprietary hardware and proprietary NOS.

And with that comes some benefits and some challenges.

Potential Benefits

• Merchant Silicon hardware bring flexibility to chose hardware and NOS’s on a per use-case basis, rather than the larger tier 1 operators dictating their requirements to vendors and then the rest of the operator community having to live with those outcomes. Overall this is immensely powerful

• The economics of using Merchant Silicon are potentially cheaper than traditional vendors.

• Running side workloads on the hardware’s compute is the norm, rather than a protracted edge case and/ or not being supported by the vendor.

Potential Drawbacks

• Bugs between the hardware and the NOS can be tricky to troubleshoot if there is a limited relationship between the NOS creator and the hardware supplier.

• Sparing the hardware is usually down to the operator using it, rather than the vendor or partner supplying it.

• The application-specific integrated circuit (ASIC) is usually Broadcom which means a clear set of use case and requirements need to be done and checked solely by the operator, rather than the checks … etc., being supported by the vendor, before settling on hardware and NOS.

Things to Watch out for

Distributed Disaggregated Chassis (DDC)

DDC is a DriveNets Proprietary ‘connector’ and whilst it is supported by the Open Compute Project, it comes at a price when evaluating ufiSpace hardware.

DDC Example

The ufiSpace S9700-23D, looks like it has 23 ports (hence the 23 in the hardware name), however those 23 ports are split up into:

• 10 x 100/400G QSFP-DD service ports

• 13 x 400G QSFP-DD fabric ports

  

So unless you are running a DDC architecture, buying this specific hardware will leave you with 13 unusable ports.

Where as the S9610-36D has:

• 36 x 40/100/200/400G QSFP-DD service ports

So be aware the ‘red’ DDC ports.

Convergence Times

Whilst there has been significant improvements in Merchant hardware over the years, and Broadcom’s Jericho line of silicon is always improving, the convergence times of Merchant Silicon solutions is still a little behind traditional vendors.

This is due to to the Mac and PC effect, as traditional vendors can tune their ecosystem, where as the same harmony can’t ben established between independently created hardware and NOSs.

Broadcom Support

Not all NOSs support all of the lower speed Qumran2 Broadcom chipset, example DriveNets only supports the Qumran2c+ chipsets, not the Qumran-UX (120Gbps), Qumran-AX (300Gbps), Qumran2u (360Gbps) and/ or Qumran2a/ Qumran-MX (800Gbps), which can be very handy in small street furniture deployments.

Broadcom StrataDNX™ vs. StrataXGS®

The key thing to note about the difference between StrataDNX™ (DNX) and StrataXGS® (XGS) is that DNX is for routing and XGS is for switching. This is because whilst XGS has some fast feed and speed combinations:

The Trident and Tomahawk chipsets that make up the XGS series are high capacity, high buffer, low memory compared with the Qumran (not shown) and Jerchio chipsets that make up the DNX series have lower top end capacity, however have good buffers and higher memory:

Summary

In my current opinion Merchant Silicon offers two distinct things over traditional vendors:

• Feed and speed combinations in 1RU form factor that aren’t as readily available in traditional vendors, so for your edge cases or for problems that don’t meet your architectural templates, there’s probably a Merchant Silicon solution for you.

• A faster path to side compute support, due to the nature of using Open Network Install Environment (ONIE).

Could Merchant Silicon Be For You?

If you have the time, patience, the hardware and NOS combination supports your use cases, and can find a supply chain that works for you and your organisation; then yes.

However don’t be lulled into the false sense of security that there’s a lower ‘price’, ‘TCO’ … etc., as the hidden costs is in your organisations R&D time to get a working solution, which you can ‘save’ with the traditional vendors (note you need a close working relationship with said vendors in order to truly leverage this).

For me it started with failure

Failure will be a big part of your professional career, whether big or almost insignificant. And unless you understand that failure isn’t the end, it’s an opportunity to learn and grow, it has the potential to consume you and dictate your path.

I had always managed to achieve at school, and for the most part learning and academia came easily to begin with, or at least I thought it did.

As a result it’s human nature to start to place ourselves on that ever increasing pedestal, which gets higher with every achievement, accomplishment and in education system every time we move up a class. Compound those factors with pride, naivety, prosperity and social pressures; no wonder we have such a powder keg of emotions in these moments.

The problem with that is that in modern society we’re not equipping these minds with the tools to deal with failure. Or reminding them to fail early and often, so that “the fall” is smaller and they grow, adapt and learn how to survive and then thrive from failure.

I was lucky, that I had people around me to pick me up when I stumbled and or fell, either academically, in life or in sports; so in those moments for others, remember that not everyone is so lucky. And if possible, be that person to help them on their way back up.

For the longest time I saw myself as having a Fixed Mindset (although I wasn't sure how to frame it at the time, also see Downstream/Finite Thinking), as I couldn't see myself learning or growing within my own environment and I thought my knowledge / ability to learn had an upper bound. However, since turning 30, engaged a Growth Mindset (although, again I wasn't sure how to frame it at the time, also see Upstream/Infinite). I noticed that actually there was ways of self reflecting and realising my achievements without requiring exams or grades or some external measurement.

So my current ‘mindset’ is to find my ‘Peter Points’ and explore past them … the difficulty is identifying the points and finding opportunities to explore.

We have to fail to move forward

Maybe hanging development on failure, adds too many conversations in the modern age, maybe the re-frame is ‘we have to grow to move forward’.

Remember that the human race would have not moved forward if there wasn’t those amongst us who tried and failed. And by gaining those experiences, as we'll talk about later, especially in diverse fields, can help you exponentially in the future.

So try new things, get your hands dirty and be prepared to say “I don’t know, however I want to find out!”.

As a youth I managed to work several jobs; helping out in a small steel fabrication workshop, delivering papers, doing several jobs on a small holdings, and working in the local shops.

The immediate take away from those experiences is that at that age, early mornings suck (and for most still will for the foreseeable future), however by doing many things and exploring several avenues of potential futures it opens the mind to possibilities and helps you understand where your limits are, where you need to improve and what your good at. It also starts the basis of a worth ethic. Yes, I said worth, not work, however you’ll need to be prepared to put some work in too.

References

Recommended Reading/Book(s)

• How to Fail at Almost Everything and Still Win Big: Kind of the Story of My Life by Scott Adams

  • NOTE: this is not an endorsement of Scott Adams, however he does make some good points in this book and calls out some uncomfortable truths.

• Antifragile: Things that Gain from Disorder by Nassim Nicholas Taleb

• Mindset - Updated Edition: Changing The Way You think To Fulfil Your Potential by Dr Carol Dweck

Now there's a few key authors who have phrased the messages that I'm trying to covey better than I ever could, so for this article I will rely heavily on quotes.

What Are You Trying To Achieve With the Customer As the 'North Star'?

So the purpose of putting the customer at the front of one's thinking when approaching any problem, is not to put them on a pedestal or try and 2nd guess them. It is used as more of a guidance, as ultimately as businesses (and especially ones that create or drive services) have to have customers.

And if the customer isn't at the front of ones thinking, then you can easily get bogged down into silos and echo chamber thinking. And that leads to (and re-enforces) the “that's not my problem attitude”, which is self serving and will lead to the destruction of creative thinking, and potentially a company.

3 barriers to upstream thinking

• Problem blindness – the belief that negative outcomes are natural or inevitable.

• A lack of ownership – all parties say That’s not mine to fix.

• Tunnelling - when juggling lots of problems people give up trying to solve them all.

So How Does Having the Customer As the 'North Star' Help?

It moves ones mindset into a more collaborative effort, and empowers people to flag problems, risks and opportunities to the business. Let me give you two very high level examples (completely fictional);

1. Bob works in the platforms/ system team and notices a bug in the code on the production WebUI. It's not an obvious bug and it only happens in some edge cases. Now by Bob might not have access or the experience to fix it, however by empowering him to flag it, then the right team can address it before the customer even notices.

2. Sally works in the software engineering team, the code she's working on does a complex set of tasks and needs access to various systems to perform it's main function, however she notices that it could be improved by accessing one additional set of data. When she tries to get the data the system holding it doesn't respond, however if she doesn't add this ‘nice to have’, then she will still complete her task. So by empowering Sally to speak up and ask for access, then the right team can help (and no dodgy back doors are created), and the customer gets an improved experience.

So How Can We Change Mindsets?

The key to this is using an upstream/Infinite thinking approach and being supported and empowered to do so.

Downstream/ Finite Thinking

Downstream thinking is defined as being reactive and typically uses system 1 ‘thinking’ (more on system 1 vs. system 2 later). Ultimately it is finite thinking, as the rules and boundaries are set and so you are ‘trapped within the box’.

Upstream/Infinite

Upstream thinking is defined as being proactive and typically uses system 2 thinking. As it is therefore infinite thinking, as (at least mentally) you keep pushing forward as if nothing is in your way and/or anything is hanging over you to hold you back.

So How Do We Avoid “The Customer Is Always Right?” Trip Hazard?

Whilst keeping an upstream/infinite mindset, it can be reframed into a service orientated mindset. This helps focus into the Kiazen as described in Setting the scene for things to come my previous article Searching for Success.

How Can We Get Customer Onboard with This?

Friction

Whilst the above has a heavy focus on how employees and business leaders can drive the positive effects, the customer too needs to be taken on this journey, even if there's no direct engagement or awareness of the businesses enlightenment.

Transparency

Keeping customers in the loop in a meaningful and educated way removed the need for unnecessary anxiety on the part of the customer. Keeping messaging clear, simple and timely goes a long way … however there will always been someone who just wants to call in for a conversation.

And finally

Recommended Reading/Book(s)

• Upstream: How to solve problems before they happen by Dan Heath

• The Infinite Game by Simon Sinek

• The Diary of a CEO: The 33 Laws of Business and Life by Steven Bartlett

• That Will Never Work: The Birth of Netflix by Marc Randolph

This will be the first of many posts in a series for the topic of Searching for Success: What I’ve Learned From Operating Networks.

Some of the elements below touch on mental health and wellbeing, and I want to be clear at the start I’ve not done any training or classes on the subject ... etc., and therefore I’m only speaking from my own personal experiences (also see Disclaimer).

A lot of the information discussed below is designed to be generic and the start of a mental framework, and thus it not meant to be a one size fits all.

So please use it as intended for your own path; a framework. And if you need additional help, please seek it.

The following material and thoughts are my own and not that of any employer either current or previous. If I’ve references or quoted someone or something I will let you know and have left links in the notes.

Are you sitting comfortably? Then I’ll begin

The reason for me writing this material is to document the initial beginnings of my own journey into dealing with conflict, failure, stress and pressure, both personally and whilst operating a telecommunications network, then trying to balance the two. Hopefully this will be a living document as I learn more throughout my journey. And give one touch point for people to add their own insights and references.

I write this from the perspective of a predominantly blue-green personality (with a flare for yellow, apparently now an ambitvert, rather than introvert or extrovert), and so not all of the examples in this article will resonate with everyone. I have tried my best to offer as wide-ranging examples as possible, and so some of them have become generalised to make sure my point(s) are clear.

Let’s start with the foundations

Starting at the deep-end for a moment, in a earlier paragraph I referred to myself as a blue-green personality, so what do I mean by that?

The personality traits come from the DISC theory and assessments created by William Moulton Marston. It was originally created as a tool to predict job performance, however it has wider reaching capabilities. The DISC theory was adapted by Thomas Erikson to make DISA, which looks at the blue personality as being more analytical, rather than compliant.

There is also the Myers-Briggs Type Indicator (MBTI) if you would like to take on further reading.

So as a predominately blue-green, I like detail, process, structure, repeating patterns and improving on the status quo. I also like to be helpful to others (hence writing this), I can be patient, enjoy routine, and until recently, was focused on (to the point of distraction at times) pleasing and appeasing others.

So for me, what’s changed? As I said before, I was a little too green in my approach to relationships, (pleasing and appeasing others) especially with authority figures and seniors, so I moved to focus on my strengths and look for my weaknesses. In doing this, I started to realise that I had a flare for yellow, and as my confidence has grown even public speaking (once completely out of bounds) has become a little more friendly … I even have a little buzz for it (as long as it's a topic I know and have a reasonable understanding of).

Therefore, I’ve realised more and more that a person can benefit from stretching themselves, even if its a learnt behaviour rather than a trait.

Setting the scene for things to come

Things won’t change over night, and they’re not going to change unless you want them to and therefore make them change. The hardest part is ‘taking the first step’, and that step can be different for everyone. What I have learnt on my journey so far is that not everyone’s path(s) start at the same point, and like a maze there’s a few dead ends, u-turns and in some cases walls/fences to climb over to find the next track/path to take you to your next calling point.

As the old saying goes:

Try as many things as you can, however learn to fail fast. That doesn't however mean rushing into things, and jumping around without direction, it just means start a clear destination, and try everything that's reasonable in getting to said destination.

The Japanese have a system; Kiazen. It was successfully adopted by Toyota in their production systems:

Whilst not possible for everyone, as not everyone has the same level of support, working towards good mental health and wellbeing rather than pushing oneself in “the grind” (to burn out), will be a better investment in the long term. Therefore, I would advocate:

• Trying to get some regular exercise.

• Taking time for yourself:

  • Mindfulness.

  • Rest.

• Never be afraid to talk or reach out.

• Start trying to be ok, with not being ok.

The key is getting to know yourself. These don’t have to be all or nothing approaches or a sudden investment of massive amounts of your time.

Unless urgent action is needed, little and often is usually the best approach. However you can set the pace for both the positive and the negative things in your life.

References

• https://www.atlastoyota.com/news/the-synergy-of-kaizen-and-toyota-lean-management

• https://en.wikipedia.org/wiki/The_Toyota_Way

Recommended Reading/Book(s)

• Surrounded by Idiots: The Four Types of Human Behaviour (or, How to Understand Those Who Cannot Be Understood) by Thomas Erikson

• The Toyota Way, Second Edition: 14 Management Principles from the World's Greatest Manufacturer by Jeffrey K. Liker

The Story

"Now if you are sitting comfortably, I'll begin …."

This all started from a young age, when I noticed a building for the first time in the town where I grew up. I'd passed it several times; however, I'd never really noticed it (granted it only drew my attention as a van exiting it nearly ran me over), and you could tell from the dirt on the walls, and the moss growing in the car park that it was a building that had been there for some time, however it wasn't used all that much.

It's a funny looking building, with (at the time) a sign next to the door of weird a figure blowing a long horn and next to it. Then on the floor, these 0.5m by 1m sections of light grey/ brown concrete inserted into the pavement with another logo, this time just the letter ‘T’, however the top of the T turned into dots as it extended to the right.

Over the proceeding weeks and months, as I passed this building on a weekly basis, I found myself noticing that it had no house or office like windows. All of the regular square windows had a grill/metal lattice on the inside, and (and the time) the pillar box like ‘slit’ windows above them (just under the roof line) with shatter proof glass (the one with the metal wire running through them). It had a flat roof, and it was dug into the slope of the hill to make sure the building was level.

Then, as per anything that gets your attention, I started to notice these two weird logos on one or two other things around my local and surrounding towns. And then eventually the penny dropped, it was the British Telecom “BT” logo(s):

1989-1992 when I first noticed them, to the present day. The logos are the trade mark of British Telecom PLC.

Finally, one day (I think it was a weekend) I jumped onto the small wall around the building and noticed a regular pattern of frames through the windows with hundreds of wires cascading off each one. It was at that moment, I became entranced with what I was looking at

Said BT (now Openreach) telephone exchange

Over the coming years I would begin to understand that the phone line in our house, and my local relatives' houses, my friends' houses all came back to the same telephone exchange, and instead of the traditional switchboard operators I'd seen in old movies, the calls were automatically switched between phone lines to make the connections.

Switchboard operators

Anolog switch as part of the Public Switching Telephone Network (PSTN)

From that, following all of the ‘tricks’ I got from a 1990's kids ‘spy book’, I ‘investigated’ into how the phone lines travelled between the exchange and peoples houses. And it didn't take long to work out that most of my town was served using drop wires, however other towns (usually cities) didn't have any overhead wiring, there was just lots of holes cut into the floor with lids on them. So how did houses without drop wires get a phone line?

Well, that was answered one day in my teens when I was visiting relatives in Sheffield and one of the pits and adjacent street cabinets was open whilst engineers was working in it. I stopped and asked a few questions, such as “what are those cables?”, “where do they come from, go to … etc.?”, and “do you need a phoneline to get the Internet?”, without missing a beat, the engineer politely answered and my fascination with connectivity bloomed into an obsession with telecommunications.

A telephone pole with circa 1950/60's terminals

A tidy copper street cabinet, this is not normal and soon to be extinct

Conclusion

So how did all of this turn into walking pit lids? Well, that comes from seeing the UK incumbent being challenged in the late 1990's into 2000's by the emerging cable operators and in large cities I started to notice other pits with different logos, such as NTL, Telewest, Yorkshire Cable … etc.

And since becoming a telecoms professional in the mid-2000's I've be watching the various telecommunications giants pile into London, Birmingham, Manchester, Leeds … etc., and now more recently since circa 2012-2015 the ‘AltNet’ operators start appearing.

Fun Facts

In the UK for PSTN the conduits that carry the telephone lines are sealed, filled with grease and then pressurised. This is to stop moisture/water ingressing into the conduit and eroding the copper cables. The air pressure is inserted by the telephone exchange and can be measured at a street cabinet. If the pressure is below a set threshold, the engineer (whilst we still have copper phone lines) is supposed to report it back to Openreach to get it investigated.

If you live close to a telephone exchange, then there’s a high probability that you’ll be on an ‘exchange only’ line, which means the equipment terminating the connectivity is inside the exchange, rather than in a street cabinet. In the mid-2010’s these were notoriously hard to upgrade from ADSL (~8Mbps) to VDSL (to-up 80 Mbps). I’m not sure how these lines will be converted to fibre, however it’s a fair guess that a new street cabinet will be installed and/ or pit (or a nearby one will be used).

The smallest exchange in the UK is on the remote Shetland Isle of Papa Stour, with just 14 homes.

External Links

• UKNOF41 - The hitchhikers guide to ducts and poles, networks and 30 years of acquisitions (youtube.com).

• https://telephone-exchanges.org.uk.

• https://www.openreach.com/news/the-final-call-for-the-traditional-telephone-exchange/.

Recommended Reading/Book(s)

• Where Wizards Stay Up Late: The Origins of the Internet Paperback by Katie Hafner and Matthew Lyon.

• Ghost In The Wires: My Adventures as the World's Most Wanted Hacker Paperback by Kevin Mitnick and William Simon.

Table of Contents

• Version Control

• Overview

  • Control Plane vs. Data Plane vs. Protocol Protection

    • Control Plane

    • Data Plane Protection

    • Protocol Protection

  • Key Considerations

• Out of Scope

• Implementation/Config

  • Control Plane

    • Assumptions

    • Variables

    • Configuration

  • BGP

    • TTL

      • Variables

      • Router 1

      • Router 2

  • Data Plane

    • Variables

    • Configuration

  • Protocol

    • Variables

    • Configuration

• References and Contributors

  • References

    • Hardening

    • TCAM

  • Contributors

• Disclaimer

Version Control

01/08/202 - v0.1 - 1Initial draft

14/07/2021 - v0.2 - Draft. Updated with Control Plane vs. Protocol Protection

16/10/2021 - v1.0 - Final revision for publication

25/11/2021 - v1.1 - Added BGP

Overview

After several years of operating different sizes of networks, I have created the following template to cover most operational circumstances. Therefore, please take the notes below as a guide, and not an exact template.

Control Plane vs. Data Plane vs. Protocol Protection

Control Plane

The key to this part of the guide is what packet is sent to the control plane (routing engine), what what is processed by the data plane (packet forwarding engine (PFE)).

As you can see from the diagram above when applying control plane protection you need to take into consideration what it is you want to permit and what will be blocked by the deny-all at the end of a policy.

Data Plane Protection

Once a protocol is turned on, you may want to block external access, as Junos will open the protocol port on all interfaces – https://kb.juniper.net/InfoCenter/index?page=content&id=KB21222&cat=SRX_SERIES&actp=LIST.

Protocol Protection

This can be summed up simply as only permit the prefixes, AS’s, communities, TLV’s … etc., that you wished to be processed by that specific protocol. And only enable the protocols you want to use.

Key Considerations

1. TCAM – does the device have enough memory to support a large or condensed ACL?

2. Purpose – What is the purpose of the ACL? Protect the control plane? Protect the interface? Protect the protocol?

3. Operational – One long policy or several policies created per process/protocol?

4. Security – Deny-all is your friend and only permit what you need (with the exception of interfaces/data plane protection).

Out of Scope

Hardening configuration for:

1. SSH

2. AAA/System login

3. NETCONF

4. Web-UI

5. Mangement interface

6. SNMP

Implementation/Config

Control Plane

Assumptions

1. All management, OSS and AAA will be done via IPv4. If you want to add IPv6, then update the IPv6 section of the guide below.

2. AAA will be done via RADIUS, so please update the guide as need to support your IPv6 architecture.

Variables

Configuration

set system default-address-selection
set system no-redirects
set system no-ping-record-route
set system no-ping-time-stamp
set system internet-options tcp-drop-synfin-set
set system internet-options no-source-quench
set system internet-options no-tcp-reset drop-tcp-with-syn-only
set system ports console log-out-on-disconnect
set system ports console insecure
set system ports auxiliary disable
set system ports auxiliary insecure
set system diag-port-authentication encrypted-password $DIAG_PORT_ENCRYPTED_PASSWORD
set system pic-console-authentication encrypted-password $PIC_CONSOLE_ENCRYPTED_PASSWORD


set interfaces lo0 unit 0 family inet filter input [ synflood-protect allow-bgp-v4 allow-ospf-v4 allow-rsvp allow-ssh allow-snmp allow-ntp allow-radius icmp-frags allow-icmp allow-traceroute tcp-established default-deny ]

set interfaces lo0 unit 0 family inet6 filter input [ allow-bgp-v6 allow-ospf-v6 icmp-frags allow-icmp allow-traceroute tcp-established default-deny ]


set policy-options prefix-list bgp-neighbors-v4 apply-path "protocols bgp group <*>-$V4 neighbor <*>"
set policy-options prefix-list bgp-neighbors-v6 apply-path "protocols bgp group <*>-$V6 neighbor <*>"

set policy-options prefix-list ipv4-interfaces apply-path "interfaces <*> unit <*> family inet address <*>"
set policy-options prefix-list ipv6-interfaces apply-path "interfaces <*> unit <*> family inet6 address <*>"


set policy-options prefix-list ospf-all-routers-v4 224.0.0.5/32
set policy-options prefix-list ospf-all-routers-v4 224.0.0.6/32
set policy-options prefix-list ospf-all-routers-v6 ff02::5/128
set policy-options prefix-list ospf-all-routers-v6 ff02::6/128
set policy-options prefix-list ntp-servers apply-path "system ntp server <*>"
set policy-options prefix-list snmp-servers apply-path "snmp community <*> clients <*>"
set policy-options prefix-list mgmt-nets $MGMT_SUBNET
set policy-options prefix-list radius-servers apply-path "system radius-server <*>"
set policy-options prefix-list localhost 127.0.0.1/32


set firewall family inet filter synflood-protect term synflood-protect from source-prefix-list bgp- neighbors-v4
set firewall family inet filter synflood-protect term synflood-protect from source-prefix-list mgmt-nets
set firewall family inet filter synflood-protect term synflood-protect from protocol tcp
set firewall family inet filter synflood-protect term synflood-protect from tcp-flags "(syn & !ack) | fin | rst"
set firewall family inet filter synflood-protect term synflood-protect then policer limit-100k
set firewall family inet filter synflood-protect term synflood-protect then accept

set firewall family inet filter allow-bgp-v4 term allow-bgp-v4 from source-prefix-list bgp-neighbors-v4
set firewall family inet filter allow-bgp-v4 term allow-bgp-v4 from destination-prefix-list ipv4-interfaces
set firewall family inet filter allow-bgp-v4 term allow-bgp-v4 from protocol tcp
set firewall family inet filter allow-bgp-v4 term allow-bgp-v4 from destination-port bgp
set firewall family inet filter allow-bgp-v4 term allow-bgp-v4 then accept

set firewall family inet filter allow-ospf-v4 term allow-ospf-v4 from source-prefix-list ipv4-interfaces
set firewall family inet filter allow-ospf-v4 term allow-ospf-v4 from destination-prefix-list ospf-all-routers-v4
set firewall family inet filter allow-ospf-v4 term allow-ospf-v4 from destination-prefix-list ipv4-interfaces
set firewall family inet filter allow-ospf-v4 term allow-ospf-v4 from protocol ospf
set firewall family inet filter allow-ospf-v4 term allow-ospf-v4 then accept

set firewall filter inet filter allow-rsvp term allow-rsvp from destination-prefix-list ipv4-interfaces
set firewall filter inet filter allow-rsvp term allow-rsvp from protocol rsvp
set firewall filter inet filter allow-rsvp term allow-rsvp then accept  

set firewall family inet filter allow-ssh term allow-ssh from source-prefix-list mgmt-nets
set firewall family inet filter allow-ssh term allow-ssh from protocol tcp
set firewall family inet filter allow-ssh term allow-ssh from destination-port ssh
set firewall family inet filter allow-ssh term allow-ssh then policer limit-10m
set firewall family inet filter allow-ssh term allow-ssh then accept

set firewall family inet filter allow-snmp term allow-snmp from source-prefix-list snmp-servers
set firewall family inet filter allow-snmp term allow-snmp from protocol udp
set firewall family inet filter allow-snmp term allow-snmp from destination-port snmp
set firewall family inet filter allow-snmp term allow-snmp then policer limit-1m
set firewall family inet filter allow-snmp term allow-snmp then accept

set firewall family inet filter allow-ntp term allow-ntp from source-prefix-list ntp-servers
set firewall family inet filter allow-ntp term allow-ntp from source-prefix-list localhost
set firewall family inet filter allow-ntp term allow-ntp from protocol udp
set firewall family inet filter allow-ntp term allow-ntp from destination-port ntp
set firewall family inet filter allow-ntp term allow-ntp then policer limit-32k
set firewall family inet filter allow-ntp term allow-ntp then accept

set firewall family inet filter allow-radius term allow-radius from source-prefix-list radius-servers
set firewall family inet filter allow-radius term allow-radius from protocol udp
set firewall family inet filter allow-radius term allow-radius from source-port radius
set firewall family inet filter allow-radius term allow-radius from source-port radacct
set firewall family inet filter allow-radius term allow-radius then policer limit-32k
set firewall family inet filter allow-radius term allow-radius then accept

set firewall family inet filter icmp-frags term icmp-frags from is-fragment
set firewall family inet filter icmp-frags term icmp-frags from protocol icmp
set firewall family inet filter icmp-frags term icmp-frags then syslog
set firewall family inet filter icmp-frags term icmp-frags then discard

set firewall family inet filter allow-icmp term allow-icmp from protocol icmp
set firewall family inet filter allow-icmp term allow-icmp from icmp-type echo-request
set firewall family inet filter allow-icmp term allow-icmp from icmp-type echo-reply
set firewall family inet filter allow-icmp term allow-icmp from icmp-type unreachable
set firewall family inet filter allow-icmp term allow-icmp from icmp-type time-exceeded
set firewall family inet filter allow-icmp term allow-icmp then policer limit-1m
set firewall family inet filter allow-icmp term allow-icmp then accept

set firewall family inet filter allow-traceroute term allow-traceroute from protocol udp
set firewall family inet filter allow-traceroute term allow-traceroute from destination-port 33434-33523
set firewall family inet filter allow-traceroute term allow-traceroute then policer limit-1m
set firewall family inet filter allow-traceroute term allow-traceroute then accept

set firewall family inet filter tcp-established term tcp-established from protocol tcp
set firewall family inet filter tcp-established term tcp-established from source-port ssh
set firewall family inet filter tcp-established term tcp-established from source-port bgp
set firewall family inet filter tcp-established term tcp-established from tcp-established
set firewall family inet filter tcp-established term tcp-established then policer limit-10m
set firewall family inet filter tcp-established term tcp-established then accept

set firewall family inet filter default-deny then log
set firewall family inet filter default-deny then syslog
set firewall family inet filter default-deny then discard


set firewall family inet6 filter allow-bgp-v6 term allow-bgp-v6 from source-prefix-list bgp-neighbors-v6
set firewall family inet6 filter allow-bgp-v6 term allow-bgp-v6 from destination-prefix-list ipv6-interfaces
set firewall family inet6 filter allow-bgp-v6 term allow-bgp-v6 from protocol tcp
set firewall family inet6 filter allow-bgp-v6 term allow-bgp-v6 from destination-port bgp
set firewall family inet6 filter allow-bgp-v6 term allow-bgp-v6 then accept

set firewall family inet6 filter allow-ospf-v6 term allow-ospf-v6 from source-prefix-list ipv6-interfaces
set firewall family inet6 filter allow-ospf-v6 term allow-ospf-v6 from destination-prefix-list ospf-all-routers-v6
set firewall family inet6 filter allow-ospf-v6 term allow-ospf-v6 from destination-prefix-list ipv6-interfaces
set firewall family inet6 filter allow-ospf-v6 term allow-ospf-v6 from protocol ospf
set firewall family inet6 filter allow-ospf-v6 term allow-ospf-v6 then accept

set firewall family inet6 filter icmp-frags term icmp-frags from is-fragment
set firewall family inet6 filter icmp-frags term icmp-frags from protocol icmp
set firewall family inet6 filter icmp-frags term icmp-frags then syslog
set firewall family inet6 filter icmp-frags term icmp-frags then discard

set firewall family inet6 filter allow-icmp term allow-icmp from protocol icmp
set firewall family inet6 filter allow-icmp term allow-icmp from icmp-type echo-request
set firewall family inet6 filter allow-icmp term allow-icmp from icmp-type echo-reply
set firewall family inet6 filter allow-icmp term allow-icmp from icmp-type unreachable
set firewall family inet6 filter allow-icmp term allow-icmp from icmp-type time-exceeded
set firewall family inet6 filter allow-icmp term allow-icmp then policer limit-1m
set firewall family inet6 filter allow-icmp term allow-icmp then accept

set firewall family inet6 filter allow-traceroute term allow-traceroute from protocol udp
set firewall family inet6 filter allow-traceroute term allow-traceroute from destination-port 33434-33523
set firewall family inet6 filter allow-traceroute term allow-traceroute then policer limit-1m
set firewall family inet6 filter allow-traceroute term allow-traceroute then accept

set firewall family inet6 filter tcp-established term tcp-established from protocol tcp
set firewall family inet6 filter tcp-established term tcp-established from source-port ssh
set firewall family inet6 filter tcp-established term tcp-established from tcp-established
set firewall family inet6 filter tcp-established term tcp-established then policer limit-10m
set firewall family inet6 filter tcp-established term tcp-established then accept

set firewall family inet6 filter default-deny then log
set firewall family inet6 filter default-deny then syslog
set firewall family inet6 filter default-deny then discard


set firewall policer limit-32k if-exceeding bandwidth-limit 32k
set firewall policer limit-32k if-exceeding burst-size-limit 15k
set firewall policer limit-32k then discard

set firewall policer limit-100k if-exceeding bandwidth-limit 100k
set firewall policer limit-100k if-exceeding burst-size-limit 15k
set firewall policer limit-100k then discard

set firewall policer limit-1m if-exceeding bandwidth-limit 1m
set firewall policer limit-1m if-exceeding burst-size-limit 15k
set firewall policer limit-1m then discard

set firewall policer limit-3m if-exceeding bandwidth-limit 3m
set firewall policer limit-3m if-exceeding burst-size-limit 15k
set firewall policer limit-3m then discard

set firewall policer limit-10m if-exceeding bandwidth-limit 10m
set firewall policer limit-10m if-exceeding burst-size-limit 625k
set firewall policer limit-10m then discard

BGP

TTL

As TTL is decremented as it passed from one device to another (accept in certain tunnelling circumstances), we can protect our P2P BGP neighbour-ships using the TTL.

Variables

Router 1

set $P2P_INTERFACE family inet filter input BGP-TTL

edit firewall filter BGP-TTL
edit term TTL
set from source-address $P2P_PREFIX
set from protocol tcp
set from ttl-except 255
set from port 179
set the discard
up
edit term then-accept
set then accept
top

Router 2

set protocol bgp group $GROUP ttl 255

or

set protocol bgp group $GROUP neighbor $NEIGHBOUR_IP ttl 255

Data Plane

Variables

Configuration

set protocols lldp interface all disable
set protocols lldp interface $LLDP_INTERFACE

Protocol

Variables

Configuration

set protocols ldp session $LDP_NEIGHBOUR_IP authentication-key $LDP_PASSWORD

set protocols rsvp interface $RSVP_INTERFACE.$UNIT authentication-key $RSVP_PASSWORD

References and Contributors

References

Hardening

• https://www.juniper.net/assets/kr/kr/local/pdf/books/tw-hardening-junos-devices-checklist.pdf

• https://www.juniper.net/documentation/jnbooks/us/en/day-one-books > https://www.juniper.net/documentation/en_US/day-one-books/TW_HardeningJunosDevices_2ndEd.zip

• https://kb.juniper.net/library/CUSTOMERSERVICE/Securing_RouteEngine2.pdf

• https://www.oreilly.com/library/view/juniper-mx-series/9781449358143/ch04s01.html

TCAM

• ACX – https://kb.juniper.net/InfoCenter/index?page=content&id=KB35363&actp=RSS and https://www.juniper.net/documentation/us/en/software/junos/network-mgmt/topics/topic-map/monitoring-and-troubleshooting-resources.html#id-troubleshooting-and-monitoring-tcam-resource-in-acx-series-routers

• EX/QFX – https://kb.juniper.net/InfoCenter/index?page=content&id=KB30953

Contributors

Disclaimer

Disclaimer

Table of Contents

• Version Control

• Prerequisites

• Implementation/Config

  • Overview

  • Variables

  • Configuration

    • VDSL

    • FTTP

  • Why DHCP Options and not JDHCPD?

• Checks/Confirmation

  • DSL Sync

  • Broadband Connection State

  • DHCP Troubleshooting

• References and Contributors

  • References

  • Contributors

• To Be Added

• Disclaimer

Version Control

VersionDateDescriptionAuthor

19/07/2021 - v1.0 - Published

20/07/2021 - v1.1 - Added IPv6 config

14/09/2021 - v1.2 - Added disclaimer

16/10/2021 - v1.3 - Updated Configuration to have VDSL and FTTP

Prerequisites

To get a Juniper SRX to work with Sky Broadband you will need:

1. Juniper SRX320 or bigger (needs the Mini PIM slot)

2. Juniper SRX-MP-1VDSL2-A (part number 750-025184)

3. Your Sky broadband username and password

   1. https://www.georgebuckingham.com/sky-fibre-router-vdsl-password/

   2. https://www.youtube.com/watch?v=qhmMh9tn_60

To establish your Sky Broadband service you need to:

1. Insert the VDSL MP into the next available slot and make sure it’s registered:

show chassis hardware 
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                CW0818AF0883      SRX320
Routing Engine   REV 0x13 650-065040   CW0818AF0883      RE-SRX320
FPC 0                                                    FPC
  PIC 0                                                  6xGE,2xGE SFP Base PIC
FPC 1            REV 19   750-025184   ACNF8918          FPC
  PIC 0                                                  1x VDSL2 Annex A

As you can see from the above, mines inserted into slot 1 of my SRX.

Implementation/Config

Overview

1. Connect the VDSL MP of your SRX directly to your phone line socket using the appropriate cable.

2. Use the following configuration to connect your SRX to your Sky broadband connection (the following assumed that the VDSL MP is in slot 1 of your SRX):

Variables

Configuration

VDSL

set interfaces $VDSL_INTERFACE description $DESCRIPTION_OF_YOUR_CHOICE$
set interfaces $VDSL_INTERFACE vlan-tagging
set interfaces $VDSL_INTERFACE vdsl-options vdsl-profile auto
set interfaces $VDSL_INTERFACE unit 101 description "$DESCRIPTION_OF_YOUR_CHOICE$"
set interfaces $VDSL_INTERFACE unit 101 vlan-id 101
set interfaces $VDSL_INTERFACE unit 101 family inet dhcp options number 61 hex-string $SKY_BROADBAND_USERNAME$|$SKY_BROADBAND_PASSWORD$
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client client-type stateful
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client client-ia-type ia-pd
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client rapid-commit
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client client-identifier duid-type duid-ll
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client req-option dns-server
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client update-router-advertisement interface $LAN_INTERFACE$
set interfaces $VDSL_INTERFACE unit 101 family inet6 dhcpv6-client update-server

FTTP

set interfaces $WAN_INTERFACE description $DESCRIPTION_OF_YOUR_CHOICE$
set interfaces $WAN_INTERFACE unit 0 description "$DESCRIPTION_OF_YOUR_CHOICE$"
set interfaces $WAN_INTERFACE unit 0 family inet dhcp options number 61 hex-string $SKY_BROADBAND_USERNAME$|$SKY_BROADBAND_PASSWORD$
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client client-type stateful
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client client-ia-type ia-pd
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client rapid-commit
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client client-identifier duid-type duid-ll
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client req-option dns-server
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client update-router-advertisement interface $LAN_INTERFACE$
set interfaces $WAN_INTERFACE unit 0 family inet6 dhcpv6-client update-server

Why DHCP Options and not JDHCPD?

The reason for using DHCP options, rather than client-identifier is because JDHCPD follows RFC2132 which by the standard adds a “00” to the packet header. However Sky is not compliant with RFC2132 and just wants a raw string for it’s username and password.

Sky CPE Output

SRX JDHCPD Output

Note that JDHCP/RFC2132 option 61 length is 29 due to the “00”

Checks/Confirmation

DSL Sync

From the CLI to confirm your DSL is in sync

show interfaces pt-1/0/0 extensive | match "profile|annex|error|bit" 
  Input errors:
    Errors: 0, Drops: 0, Policed discards: 2, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, Resource errors: 0
  Output errors:
    Carrier transitions: 1, Errors: 0, Drops: 0, Aged packets: 0, MTU errors: 0, Resource errors: 0
    Modem status  : Showtime (Profile-17a)
    VDSL profile  :     Auto    Annex B
    Bit rate (kbps)   :                32263          0        4876          0
    Flow error statistics (Packets dropped due to): 
      Incoming NAT errors:               0
      User authentication errors:        0
    Flow error statistics (Packets dropped due to): 
      Incoming NAT errors:               0
      User authentication errors:        0

You can also confirm the SNR with:

show interfaces pt-1/0/0 extensive | match "vtu|db" 
  VDSL Chipset Information:               VTU-R                  VTU-C       
  VDSL Statistics:                        VTU-R                  VTU-C        
    Attenuation (dB)         :              0.0                    0.0
    Noise margin (dB)        :              5.5                    6.0
    Output power (dBm)       :              2.0                    2.0

Broadband Connection State

As this is DHCP based, rather than PPP, to confirm that your SRX has connected to your Sky broadband you can use the following commands:

show dhcp client binding 

IP address        Hardware address   Expires     State      Interface
AAA.BBB.CCC.DDD   64:c3:d6:AA:BB:CC  2282        BOUND      pt-1/0/0.101


show dhcpv6 client binding 

IP/prefix                       Expires     State      ClientType    Interface       Client DUID
2a02:c7f:AAAA:BBBB::/56         2043        BOUND      STATEFUL      pt-1/0/0.101    LL0xd-64:c3:d6:AA:BB:CC

or

show interfaces terse
<SNIP>
pt-1/0/0.101            up    up   inet     AAA.BBB.CCC.DDD/22
                                   inet6    fe80::66c3:d600:65c2:142a/64
pt-1/0/0.32767          up    up  
<SNIP>
irb.10                  up    up   inet     192.168.0.1/24  
                                   inet6    2a02:c7f:AAAA:BBBB::1/64
                                            fe80::66c3:d600:ac2:14b0/64

or

show route table inet.0 protocol access-internal | match /0     
0.0.0.0/0          *[Access-internal/12] 02:47:55, metric 0
                    >  to AAA.BBB.CCC.1 via pt-1/0/0.101


show route table inet6.0 protocol access-internal               

inet6.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

::/0               *[Access-internal/12] 02:32:11
                    >  to fe80::a2f3:e4ff:fe5d:9230 via pt-1/0/0.101
2a02:c7f:AAA:BBB::/56
                   *[Access-internal/12] 02:32:12
                       Reject

DHCP Troubleshooting

The following links cover what’s needed to troubleshoot your SRX as a DHCP client:

• https://www.juniper.net/documentation/us/en/software/junos/dhcp/topics/ref/command/show-dhcp-client-binding-srx.html

References and Contributors

References

The following articles helped me along the way:

• Making a modern SRX work with Sky VDSL – https://untrust.zone/making-a-modern-srx-work-with-sky-vdsl/

• JUNIPER SRX on SKY Fibre Broadband – http://www.ddrcomputing.co.uk/juniper-funk/junipersrx

• IPv6 on SRX – https://blog.netpro.be/ipv6-on-juniper-srx-prefix-delegation-dhcpv6/

Contributors

• James Bensley

• Richard Patterson

• Tim Cooper

To Be Added

Disclaimer

Disclaimer