▲

Emergency Network Recovery & Infrastructure Stabilization

Network

Client Industry: Hospitality / Hotel

Role: Lead Systems Administrator (Remote Lead & Technical Coordinator)

Key Technologies: Meraki router, Managed Aruba PoE Switches, VLAN Isolation, Aruba Wireless Access Points, Layer 1/Layer 2 Diagnostics

Overview

Responded to a critical, multi-system infrastructure failure at a extremely remote hotel property involving a total outage of the primary IP security camera system and guest Wi-Fi networks. The environment consisted of four core switches managing eight distinct VLANs, serving both in-room access points and high-density common-area wireless access points.

Spaghetti

Technical Challenges & Discovery

  • Hardware Failure & Physical Risk: The main PoE switch governing the camera network suffered catastrophic failure. Diagnostic investigation revealed that external PoE injectors had been improperly patched directly into active PoE switch ports during previous onsite troubleshooting, creating power/loop hazards.

  • Outdated Network Documentation: A pre-configured replacement switch was prepared off-site using existing network maps and documented diagrams. Upon deployment, physical-layer discrepancies revealed extensive undocumented changes, rearranged cables and edge devices had been moved or added over time without any change-management tracking.

  • Gateway DNS Failure: Investigation into the guest Wi-Fi drop revealed that while physical and data link layers were functional, the gateway security appliance had stopped proxying DNS requests across the guest and production VLANs, leaving clients unable to resolve hostnames.

Action & Execution

  • Physical De-risking: Safe removal of all redundant external PoE injectors to prevent electrical feedback and protect replacement hardware.

  • Remote-Guided Layer 1/2 Tracing: Orchestrated an overall 4-hour, step-by-step physical-to-logical line audit working synchronously with an on-site technician. Systematically traced run terminations across four switch bays to map actual traffic routes.

  • DHCP Scope Reconfiguration:: Updated DHCP server configuration across the affected guest and production VLAN's on the router for each affected profile, overriding the local gateway relay upstream and adding explicit, reliable custom DNS servers (e.g., 1.1.1.1 / 8.8.8.8).

Business Impact & Value

  • 100% Service Restoration: Fully restored functionality to all IP security camera streams and guest Wi-Fi access points across the property.

  • Preventative Hardware Safeguarding: Neutralized back-feed hazards caused by misconfigured placement of PoE equipment, preventing future hardware burnout.

  • Updated Infrastructure Baseline: Corrected and re-documented the facility’s physical port mappings and VLAN topology to eliminate downtime during future maintenance windows.