Out-of-band (OOB) management is a way to manage a server through a separate, dedicated management channel instead of through the server’s operating system and production network (which is in-band management).
Key points:
OOB works even when the OS has crashed, hasn’t been installed yet, or the server is powered off, as long as the server is plugged in, because the management controller runs on standby power.
It is provided by a Baseboard Management Controller (BMC), a small dedicated processor built into the server motherboard with its own network port, IP address, firmware and web interface.
Vendor implementations:
Dell: iDRAC (Integrated Dell Remote Access Controller)
HPE: iLO (Integrated Lights-Out)
Lenovo: XClarity Controller (XCC), formerly IMM
Cisco: CIMC (Cisco Integrated Management Controller)
Supermicro and others: generic IPMI/BMC
Standards:
IPMI (Intelligent Platform Management Interface): the older industry standard. Uses UDP port 623, and is managed with tools such as ipmitool.
Redfish: the modern RESTful API standard (HTTPS/JSON) for server management and automation, replacing IPMI.
What OOB management can monitor: temperatures, fan speeds, voltages, power supply status, drive health and system event logs (SEL). It can send alerts by email, SNMP traps or syslog.
Change default credentials immediately, since BMCs often ship with known defaults.
Put management ports on a separate, isolated management network or VLAN, never on the internet.
Keep BMC firmware updated, because BMC vulnerabilities give attackers full control of the hardware.
Use HTTPS/TLS, strong passwords, role-based accounts, and directory integration (LDAP/AD) where possible.
Disable unused services, such as IPMI over LAN if Redfish or the web interface is used, because IPMI 2.0 has known authentication weaknesses.
Log and audit all management access.
Also called virtual media.
Mounts a file or drive from the administrator’s computer, such as an ISO image, a USB drive or a folder, so that the server sees it as a locally attached CD/DVD or USB drive.
Uses:
Installing or reinstalling an OS remotely from an ISO.
Booting diagnostic or recovery tools.
Applying firmware update images.
Copying driver files to a server that has no network connection.
Can also include remote viewing of the server’s storage: drive status, RAID configuration, and controller and drive health through the BMC.
Virtual media performance depends on the network link between the administrator and the BMC, so large ISOs may load slowly.
Also called a virtual console or remote KVM through the BMC.
Shows the server’s video output and lets the administrator use the keyboard and mouse remotely, exactly as if they were standing in front of the server.
What it allows:
Watching POST, entering BIOS/UEFI setup, configuring RAID controllers and viewing boot errors.
Troubleshooting a frozen OS or a blue/purple screen.
Working before an OS or network stack is running.
Modern consoles run in the browser using HTML5. Older versions needed Java or ActiveX.
Serial over LAN (SOL): redirects the server’s text-based serial console across the network through the BMC, typically using ipmitool sol activate. Useful for Linux servers and text-mode consoles.
Features may include screen recording or capture of the last crash screen, and multiple sessions shared between users.
Some advanced console features require a license, for example iDRAC Enterprise or iLO Advanced.
Controls server power through the BMC without physical access.
Options:
Power on
Graceful shutdown: sends an ACPI signal so the OS shuts down cleanly.
Power off (hard / forced): like holding the power button. Risks data loss, so use only when the OS is unresponsive.
Reset (warm reboot)
Power cycle (cold reboot): off, then back on.
Example IPMI commands: ipmitool -I lanplus -H <BMC-IP> -U <user> chassis power on | off | soft | cycle | status
Use cases:
Restarting a hung server at a remote site.
Powering servers on for scheduled maintenance.
Shutting down to save power or during an emergency.
Wake-on-LAN (WoL) can also power a server on remotely by sending a “magic packet” to the NIC. However, WoL is in-band and can only power a machine on. OOB management is much more capable.
Switched PDUs can power-cycle an outlet remotely as a last resort.
A hardware KVM switch that sends keyboard, video and mouse signals over an IP network.
Administrators connect through a web browser or client software to see and control the attached servers.
Differences from a BMC virtual console:
Vendor-neutral: works with any server or device that has video and USB/PS2 ports, including older servers or servers without a licensed BMC.
It is an external device, so it does not depend on the server’s own management controller.
One IP KVM can manage many servers, typically 8–64 ports, often through small adapters called dongles or server interface modules.
Features:
BIOS-level access.
Virtual media.
Multi-user access.
Encryption.
Authentication (local, LDAP/AD, RADIUS).
Audit logs.
Place it on the management network and secure it like any other OOB device.
Local administration means managing a server physically, in person at the rack. It is needed when remote access fails, during initial setup, and for hardware repair.
A KVM switch connects one keyboard, monitor and mouse to multiple servers so administrators do not need a separate console for each.
Usually a rack-mounted 1U console drawer with a fold-out LCD, keyboard and touchpad, connected to a KVM switch.
How it works:
Each server connects through cables or dongles that carry VGA/HDMI/DisplayPort video and USB (or legacy PS/2) keyboard and mouse.
Switch between servers using hotkeys, buttons, or an on-screen display (OSD) menu that lists servers by name.
Larger environments can cascade (tier) several KVM switches together.
Best practices:
Label ports and keep server names updated in the OSD.
Set an OSD password and auto-logout.
Log out of the server OS after use, because the next person at the KVM sees whatever session was left open.
A mobile cart carrying everything needed to connect directly to a server that has no console, or that cannot be reached remotely or through a KVM.
Typical contents:
Monitor, keyboard and mouse.
Video and USB cables and adapters (VGA, HDMI, DisplayPort, USB-C).
A power strip or extension cord.
A laptop with terminal software.
Console/rollover cables and USB-to-serial adapters.
Spare patch cables.
Bootable USB drives with OS installers, diagnostics and firmware.
Basic tools, an ESD wrist strap, and labels.
The cart is wheeled up to the server and plugged directly into the server’s front or rear ports.
Used when:
A server has crashed and is unreachable over the network or through OOB.
Configuring a new server for the first time.
Servers in the row are not connected to a KVM.
A software-based console for administering a server or VM without a physical monitor connected directly to the hardware.
Examples:
Hypervisor consoles: the VM console in VMware vSphere/ESXi, Hyper-V Manager’s VMConnect, and Proxmox/KVM virt-manager. These let administrators see a VM’s “screen”, including its BIOS and boot, even when the guest OS network is down.
The BMC’s virtual console (iDRAC, iLO, XCC), when it is reached locally on the management network.
Front-panel management on some servers: a small LCD panel, or a direct USB/NFC management port that lets a laptop or phone connect to the BMC at the rack (for example iDRAC Direct USB or HPE iLO service port).
Advantages: no extra physical hardware, and the console is available as soon as the VM or server powers on.
Text-based console access through a serial port. This is the most basic and dependable form of console access, and it works even when there is no video or network.
Common on network devices (switches, routers, firewalls), storage arrays, UPS units, PDUs, and Linux/UNIX servers.
Connectors and cables:
DB-9 (RS-232) serial port on the server.
RJ-45 console port on many network and storage devices, used with a rollover (console) cable, which is RJ-45 to DB-9.
USB-to-serial adapter: needed because most modern laptops have no serial port. Requires a driver and appears as a COM port in Windows or /dev/ttyUSB0 in Linux.
USB console ports (USB-mini or USB-C) on newer devices.
Common terminal settings (must match the device):
9600 baud, 8 data bits, no parity, 1 stop bit (8-N-1), no flow control. Some devices use 115200 baud.
Terminal emulation software:
PuTTY, Tera Term or SecureCRT on Windows.
screen or minicom on Linux and macOS, for example screen /dev/ttyUSB0 9600.
Console redirection: a BIOS/UEFI setting that sends POST and boot output to the serial port. Combined with Serial over LAN, this gives remote text console access.
Console servers (terminal servers): network devices with many serial ports. Administrators can reach the serial consoles of many devices over the network. This is a form of out-of-band management.
Direct, physical connections to a device’s console or management interface for local administration.
Types of console connection:
Video and USB on the front or rear of the server, connected to a monitor, keyboard, KVM or crash cart.
Serial console port (DB-9, RJ-45 or USB), connected through a rollover cable or console server.
Dedicated management port (the BMC’s RJ-45 port), connected directly to a laptop with a crossover or standard patch cable for local BMC access.
Front-panel USB management port, for direct BMC access from a laptop at the rack.
Use console connections to:
Perform initial configuration, such as setting the BMC IP address or BIOS settings.
Recover from network misconfigurations, for example when a firewall rule or IP change has locked out remote access.
Watch boot messages and troubleshoot.
Console access gives deep control, so secure it physically with locked racks and controlled data center access, and require authentication.
Firmware is low-level software stored on a hardware component’s non-volatile memory. It controls how that component operates.
Server components with firmware:
BIOS/UEFI (system firmware)
BMC (iDRAC, iLO, XCC)
RAID controllers and HBAs
NICs, CNAs and Fibre Channel HBAs
Drives (HDD and SSD)
Backplanes and drive expanders
Power supplies
CPLD (Complex Programmable Logic Device)
GPUs
Tape drives
Fix bugs and stability issues.
Patch security vulnerabilities, such as BMC and CPU microcode flaws.
Add support for new hardware (newer CPUs, larger drives, new OS versions).
Improve performance.
Meet vendor support requirements. Vendors often require current firmware before they will troubleshoot an issue.
Read the release notes, including fixes, known issues, prerequisites and upgrade order dependencies. For example, some vendors require the BMC to be updated before the BIOS.
Check compatibility with the HCL, the OS or hypervisor version, and the drivers. Firmware and drivers often need to be updated together as matched versions.
Follow change management: get approval and schedule a maintenance window.
Back up configurations, such as BIOS settings, RAID configuration and BMC settings, plus data.
Verify the download by getting firmware only from the vendor and checking its hash or digital signature.
Test in a lab or staging environment first when possible.
Ensure stable power using a UPS and redundant PSUs. Never interrupt a firmware flash, because a power loss or reboot during the update can “brick” the component.
Update one server at a time in a cluster, and migrate workloads away first.
Verify after the update that the new version is installed and the system boots normally, and check the logs.
Keep a rollback plan, such as a previous firmware image or dual-bank firmware for recovery.
Document the versions installed.
Dell: Lifecycle Controller, Dell System Update (DSU), and OpenManage Enterprise.
HPE: Service Pack for ProLiant (SPP), Smart Update Manager (SUM), and iLO Amplifier/OneView.
Lenovo: XClarity Administrator and XClarity Essentials.
Updates can also be done through the BMC web interface, by booting a bootable ISO or USB, or by running OS-level update packages.
Drive maintenance tasks:
Monitor drive health:
S.M.A.R.T. (Self-Monitoring, Analysis and Reporting Technology) data includes reallocated sectors, pending sectors, temperature and SSD wear level.
RAID controller and BMC alerts report predictive failure warnings.
Watch for drive status LEDs. Amber or blinking amber usually means a failure or predicted failure, and green means healthy or active. Check the vendor’s LED documentation.
Proactively replace drives that report predictive failure, before they actually fail.
Identify the correct drive before removal:
Use the locate/identify LED function in the RAID or BMC utility to blink the drive.
Check the slot number in the management software.
Pulling the wrong drive from a degraded array can destroy the array.
Match replacement drive specifications:
Same interface (SAS, SATA or NVMe) and same media type (HDD or SSD).
Equal or larger capacity.
The same RPM or performance class.
Same form factor (2.5-inch or 3.5-inch) and the correct caddy/sled.
Vendor-certified part and supported firmware.
Monitor the rebuild after replacement. Performance is reduced and the array is vulnerable until the rebuild completes, so avoid heavy work during this time.
Configure hot spares so rebuilds begin automatically.
Keep drive firmware current, since drive firmware bugs can cause failures or data corruption.
Use ESD protection when handling drives, and handle HDDs gently because they are sensitive to shock.
Dispose of failed drives securely: wipe, degauss or physically destroy them according to policy, especially drives that held sensitive data. Some organizations use a “keep your drive” support agreement so that failed drives are not returned to the vendor.
Hot-swappable components can be removed and replaced while the server is powered on and running, without shutting it down and without interrupting service. This supports high availability and reduces downtime.
Related terms:
Hot swap: replace a component while the system is running with no special preparation. The system handles it automatically.
Hot plug: add or remove a component while the system is running, but the OS may need to be told first. For example, you “prepare for removal” in software, or press an attention button.
Cold swap: the server must be powered off before replacing the component. Examples: CPU, memory (in most servers), motherboard, and most PCIe cards.
Hot-swapping works only if:
Redundancy exists, such as a redundant PSU, an N+1 fan or a RAID-protected drive, so the system keeps running without the component.
The hardware, backplane and OS support hot swap for that component.
General rules:
Confirm the component is actually failed by checking LEDs, BMC logs and management software.
Have the correct replacement part ready before removing the failed part.
Replace one component at a time.
Use ESD protection.
Verify that the new part is recognized and that its status returns to healthy.
Drives in front-loading hot-swap bays sit in caddies/sleds/carriers that plug into a backplane.
Replace a failed drive in a redundant RAID array (RAID 1, 5, 6 or 10) while the server is running, and the rebuild starts automatically.
Replacing a drive in a RAID 0 array or a non-redundant volume loses that data even if the bay is hot-swappable.
Some systems require you to mark the drive offline or “prepare to remove” first. This is especially true for NVMe hot plug, which needs OS support.
Drive cages are modular bays or enclosures that hold several drives and their backplane.
On some servers, a whole drive cage can be added or replaced, for example to expand capacity or to convert SAS/SATA bays to NVMe.
In external storage enclosures and JBODs, entire drive drawers or controller and I/O modules are often hot-swappable when redundant modules exist.
Many internal server cage changes still require the server to be powered off, so check vendor documentation.
Hot-plug PCIe is supported on some high-end servers and storage systems. It requires:
Motherboard and slot support.
OS support.
Usually an attention button or LED for each slot, or a software command to power the slot down before removal.
Examples: NICs, HBAs and NVMe add-in cards in high-availability systems, and I/O modules or mezzanine cards in blade chassis.
Most standard rack servers do not support hot-swapping PCIe cards. Power down first unless the documentation explicitly says otherwise.
After replacement, verify the driver loads and the configuration is restored. For example, a new HBA has a new WWPN, so SAN zoning may need to be updated. A new NIC has a new MAC address, which can affect DHCP reservations or licensing.
Servers with redundant PSUs (1+1 or N+1) allow one PSU to be replaced while the other carries the load.
Steps:
Confirm which PSU has failed using its LED and the BMC log.
Confirm the other PSU is healthy and can carry the full load alone.
Unplug the failed PSU’s power cord, release the latch, and slide it out.
Insert the replacement, reconnect power, and check that the LED turns green.
Use an identical PSU with the same model, wattage and efficiency rating. Mixing PSUs is often unsupported and can cause warnings or failures.
Plug the replacement into the correct PDU feed (A or B) to maintain power redundancy.
Servers use redundant, hot-swappable fan modules, typically in an N+1 configuration.
When a fan fails, the remaining fans spin faster to compensate, and the BMC raises an alert.
Replace failed fans quickly:
While a fan module is removed, the server may be under-cooled, and some servers trigger a thermal shutdown if a fan is out too long. Have the replacement in hand before removing the failed fan.
Keep the chassis lid closed except while swapping, because airflow depends on the lid. Some fans are accessible only with the lid off.
Also check air filters, blanking panels and airflow paths, and keep dust under control.
The system firmware initializes the hardware at power-on, performs POST (Power-On Self-Test), provides configuration settings, and hands control to the bootloader and OS.
The older firmware standard, from the original IBM PC architecture.
Uses the MBR (Master Boot Record) partition scheme. Boot disks are limited to 2 TB with up to 4 primary partitions.
16-bit mode, with a text-based, keyboard-only interface.
Slower boot, and limited security features.
Replaces BIOS and is the default on all modern servers.
Features:
Uses GPT (GUID Partition Table), which supports boot disks larger than 2 TB and up to 128 partitions in Windows.
Secure Boot: only allows bootloaders and drivers signed with trusted keys, which protects against rootkits and bootkits.
Faster boot times.
Graphical interface with mouse support.
Network boot, including PXE over IPv4/IPv6 and HTTP boot.
32- and 64-bit operation.
Modular drivers.
CSM (Compatibility Support Module): emulates legacy BIOS for older OSs. Secure Boot requires CSM to be disabled (UEFI-only mode).
Changing between legacy BIOS and UEFI mode on an installed system can stop it from booting, because the disk partition scheme (MBR or GPT) must match the boot mode.
Boot order: the device sequence (disk, PXE, USB, virtual media). Changed for OS installs or network boots. A one-time boot menu is available through a key such as F11 or F12, depending on the vendor.
Virtualization:
Enable Intel VT-x / AMD-V for hypervisors.
Enable VT-d / AMD-Vi (IOMMU) for device passthrough.
CPU settings: Hyper-threading/SMT, turbo boost, and C-states.
Power/performance profiles: performance vs. power-saving vs. OS-controlled. High-performance profiles are common for latency-sensitive workloads.
Memory settings: memory mode (optimizer, mirroring, sparing) and memory testing.
Security:
Set administrator/supervisor and user (boot) passwords.
Enable the TPM (Trusted Platform Module), used for BitLocker, measured boot and attestation.
Enable Secure Boot.
Disable unused ports and devices, such as USB and serial ports, and unused NICs.
Enable intrusion detection (chassis open alerts) where available.
Integrated devices: enable or disable onboard NICs, RAID or SATA mode (AHCI vs RAID), and SR-IOV.
Date and time: must be correct for logs, Kerberos and certificates.
Embedded configuration utilities in the firmware:
RAID controller setup.
NIC/iSCSI boot configuration.
Vendor lifecycle tools for OS deployment, firmware updates and diagnostics. Examples include Dell Lifecycle Controller (F10) and HPE Intelligent Provisioning (F10) with the System Utilities (F9).
Back up or export BIOS/UEFI configuration before changes or updates. Many BMCs can export server configuration profiles, which also helps deploy identical settings to many servers.
Follow the firmware upgrade best practices above.
Do not lose power during a BIOS flash.
Many servers have dual-bank/backup BIOS or a recovery mode that can restore firmware if a flash fails.
BIOS updates often include CPU microcode fixes, for example for Spectre/Meltdown-class vulnerabilities, and add support for new CPUs and memory.
BIOS settings may reset to defaults after an update, so verify them afterward.
BIOS/UEFI settings and the real-time clock are kept in memory powered by a CMOS battery, typically a CR2032 coin cell.
Symptoms of a dying battery: the system clock resets, BIOS settings revert to defaults, and “CMOS checksum error” messages appear at boot.
Settings can be reset by:
Choosing “load defaults” in setup.
Using a clear CMOS jumper on the motherboard.
Removing the battery briefly while the server is unplugged.
A password reset jumper clears a forgotten BIOS password. This is one reason physical security matters.
POST checks the CPU, memory, storage controllers and video before booting.
Errors are reported through:
Beep codes.
Front-panel LEDs or LCD error codes.
On-screen messages.
The BMC System Event Log (SEL).
Look up codes in the vendor documentation, then reseat or replace the failing component.
Install an OS on a server at a remote site with no staff on site → mount the ISO through virtual media (remote drive access) using the BMC, then use the remote console.
Remote server is hung and unresponsive to SSH/RDP → use OOB management (iDRAC, iLO, IPMI) for remote console and a power cycle.
Need BIOS-level remote access to older servers from different vendors without licensed BMCs → IP KVM.
Server is unreachable over the network and isn’t on a KVM → wheel up a crash cart.
Configure a switch, firewall or UPS with no IP address set yet → serial console connection using a rollover or USB-to-serial cable, a terminal emulator, and settings of 9600 8-N-1.
Need to view a VM’s boot screen when its network is down → hypervisor virtual administration console.
Securing BMCs → isolated management VLAN, changed default passwords, and updated BMC firmware.
Firmware update guidance → read release notes, follow dependency order, back up configuration, use a maintenance window, never interrupt the flash, and update drivers to matching versions.
Drive showing amber predictive-failure LED in a RAID 5 array → use the locate LED to confirm the slot, hot-swap it with an identical or larger drive, and monitor the rebuild.
Failed PSU in a server with redundant power → hot-swap it with an identical model and connect it to the correct PDU feed.
Failed fan → hot-swap it quickly to avoid thermal shutdown.
New HBA installed and the SAN LUN is no longer visible → update zoning or LUN masking with the new WWPN.
OS needs to boot from a 4 TB disk with Secure Boot → UEFI mode with GPT, and CSM disabled.
Server clock keeps resetting after power loss → replace the CMOS battery.
Hypervisor won’t install or run VMs → enable VT-x / AMD-V in BIOS/UEFI.