EVPN/MPLS resilient network enabled in 11 data centers in Sofia, Bulgaria.
Looking Glass allows real-time monitoring of all routing information learned from Route Servers and provides a complete view of routing from BIX.BG's perspective. The information is public and can be useful to anyone optimizing their networks and routes or tracking down a technical issue.
Looking Glass is a useful tool that allows members to check:
- Which are all possible routes to a certain prefix and which is the shortest (best) of them, that is announced to the other members;
- Whether their announcements are received and with what BGP Communities;
- Whether there are filtered announcements and, if so, what the reason is;
- Whether BGP Blackholing is active;
- Limit of prefixes, etc.
Quality of Service
Our Quality of Service (QoS) policy determines the priority order of packets and is tailored to the specific characteristics of different types of services and traffic.
We use only Juniper Networks equipment that has large hardware buffers (100ms) on every single port in the network regardless of its speed. This way, during short-term overloads, we can ensure that useful packets are buffered and served instead of being lost.
DDoS Mitigation helps Public Peering members to significantly limit the negative effect on their networks in case of large DDoS attacks against them.
Our technical solution is focused on delivering the useful traffic to the member even if their port is overloaded as a result of a massive DDoS attack and packet loss is inevitable. In this way, members do not need to use higher speed ports just to be sure that their connection to BIX.BG will not be overloaded in case of a large DDoS attack against them.
Border Gateway Protocol (BGP) Blackholing enables Public Peering members to instruct BIX.BG to block traffic to a specified prefix (IP address or network), signaling via BGP4 announcement through already established sessions with Route Servers (RS).
BGP Blackholing is used in large DDoS attacks that overload the networks of the members and interfere with their normal functioning.
Dropping the packets (to Blackoholed host/network) inside the BIX.BG network relieves the member's network and allows the normal functioning of the unattacked and unblocked network resources.
One of the fundamentals underlying the architecture of the BIX.BG network is to avoid any possibility of Layer 2 loops occurring, which could overload the entire supporting network, including clogging the members' ports with member traffic.
Implementation: To avoid loops in the BIX.BG network, the following basic mechanisms are used:
- Pure L3 IP/MPLS at the infrastructure level (underlay);
- EVPN-MPLS – loop-free L2 service (overlay) based on the split horizon principle and using BGP signaling;
- MAC Pinning for the Public Peering service – A MAC address learned on a given port cannot be learned on another client port, and any attempt to receive traffic with the same MAC address is ignored. In this way, the migration of client MAC addresses is controlled (without expired aging-time) and the creation of „customer triggered“ loops is effectively prevented;
- Multicast VPN (MVPN) – a reliable and secure technology for transporting Multicast applications (L3 overlay) using the same principles as BGP/MPLS VPNs for complete infrastructure isolation (L3) of all Multicast sources and receivers.
Points of Presence
Telepoint Sofia Center 122 Ovtche pole Str. Sofia 1303, BulgariaTelepoint Sofia East 8 Asen Yordanov blvd. 1592 Sofia, Bulgaria
Equinix SO1 10, 5030 Str., Druzhba-1 Sofia 1592, BulgariaEquinix SO2 33 Nedelcho Bonchev Str. Sofia 1528, Bulgaria
Evolink Data Center Sofia 1 25-А, Akad G. Bonchev Str. Sofia 1113, BulgariaEvolink Data Center Sofia 2 16-V, Barzaritsa Str. Sofia 1618, Bulgaria
Neterra Sofia Data Center and Sofia Teleport
VIVACOM DC Sofia 2 8 Asen Yordanov blvd. 1592 Sofia, Bulgaria
Novatel 2, Kukush Str. Sofia 1309, Bulgaria
S3 Company 63 Shipchenski Prohod blvd. Sofia 1574, Bulgaria
CETIN Bulgaria Business Park Sofia, Bulgaria
Concept and Design
The main goal of the network is to ensure maximum service reliability, predictable behavior and ease of service. For this purpose, an EVPN/MPLS architecture is used, based on the same type (from the point of view of functionality) of Juniper Networks equipment, the differences between the different models being only in the number of their ports. In this way, the services and the way they are set up and served are completely identical regardless of location and specific equipment.
The network is designed on the basis of the following fundamental principles:
- Decentralized EVPN/MPLS architecture to avoid Single point of failure and Layer 2 loops (infrastructure/underlay and service/overlay);
- Each network device is connected to at least two others, which are preferably located in different locations;
- Any network device is a transit host between any two other devices in case this is the shortest path between them;
- All devices are connected through dark fibers, and upon reaching 35% load, new connections are activated;
- Dark fibers to one direction are on different physical routes and are leased by different operators;
- If any dark fiber (or device along the way) fails, the switching mechanisms are activated automatically in less than ten milliseconds;
- Service contracts with Smartcom Bulgaria, including hardware replacement, software troubleshooting and escalation to Juniper Networks;
- Spare equipment in stock, which is also used for tests and simulations before the changes are integrated into the network.
EVPN/MPLS protected architecture
End-to-end Juniper network equipment
100G-enabled at any PoP
BIX.BG uses the so-called system of Route Servers (RS) (detailed in RFC7947) to implement the exchange of BGP4 announcements between members. Each member only needs to initiate a BGP4 session with the RSes to receive announcements for all connected peers in BIX.BG.
IPv4 address: 184.108.40.206
IPv6 address: 2001:7f8:58::3d35:0:1
IPv4 address: 220.127.116.11
IPv6 address: 2001:7f8:58::3d35:0:2