Firmware Updates Across Hundreds of Servers: Centralized Management


Cover Image: Centrally Managing Firmware Updates Across a Server Fleet

Firmware updates across a fleet of servers sound like routine work, until the number of servers exceeds 50 or 100. At that scale, it’s primarily the amount of manual effort involved that determines whether a known security vulnerability remains unpatched.

Short answer

Once you have about 50 servers, firmware updates require a dedicated process: an inventory of all versions, a staging group with representative hardware, a fixed maintenance window, and a prepared rollback plan. Redfish standardizes the update interface across vendors, but does not replace the need to verify which version is suitable for which server type.

Why Manual Firmware Updates Have Their Limits

HPE clearly explains the difference. Manually onboarding more than 100 servers takes hours or even days, whereas the same task can be completed with a single command using iLOrest's bulk automation (HPE, 2026). The reason why firmware maintenance in growing server fleets requires a planned process lies between these two figures.

Updating a single server is quick. You log in to the management interface, upload the new firmware file, start the process, wait, and check the result. With ten servers, you can still do this manually. With a hundred or several hundred servers, this routine turns into a full-time project in its own right, where every manual step is an additional source of error.

10.000 $

per year and per site is what truck-roll costs can reach, that is, the cost of sending technicians to the site in person when no remote access is set up.

Vertiv, 2022

Redfish as a Common Language for Firmware Updates

For automation to work across different manufacturers, a common interface is needed. The DMTF standard Redfish has standardized firmware update services since version 2016.2, usable through the same API structure at every manufacturer that supports it. Anyone running a mixed fleet benefits from this directly. Instead of maintaining a separate update script for each manufacturer, most of the logic can be handled through a single common interface. How the switch from IPMI to that interface works is described in the article on the IPMI-Redfish Migration.

This reduces the maintenance effort required for automation, but does not replace content planning. Each manufacturer releases firmware on its own schedule, with its own release notes and known issues. A common interface does not mean that every firmware version is automatically suitable for every server type.

Update orchestration over long distances

For sites spread out over long distances, there is an additional technical consideration. How much data traffic actually travels over the wide area network (WAN) for an update? SynergyCP, for example, orchestrates provisioning over the WAN in such a way that the actual data traffic remains locally at the site, and only control commands are routed through the central instance (SynergyCP, 2026). For firmware files, which can be several hundred megabytes in size depending on the server generation, this difference makes a significant impact when site connectivity is limited.

The Process from Inventory Count to Rollback

Four Steps of a Robust Firmware Process
Step Contents What happens without it
Inventory Version, hardware generation, and intended use for each server It remains unclear which systems are still affected by a known vulnerability
Staging Group Test a small, representative subset first A single error immediately affects the entire fleet
Phased Rollout Group by group, rather than all at once No stop is possible if any irregularities are detected
Rollback Plan Previous version available; procedure documented via the BMC A failed update leads to a prolonged outage

Without an inventory, it is impossible to prioritize or determine which systems are still affected by a known security issue. The staging group should indeed cover all hardware variants currently in use, not just the latest models. A fixed maintenance window is just as essential as the rollback plan.

Even the best preparation doesn't rule out the possibility that...A single device may fail to restart as expected after an update, perhaps due to an incompatible hardware combination that went unnoticed during staging. Those who have a centralized view of the fleet can quickly identify such outliers and address them specifically, rather than halting the entire rollout.

Retrieve the firmware version for the entire fleetWe provide an inventory across all locations and a recommendation for staging groups and rollout order. Initial consultation: 30 minutes, free of charge, response within one business day.

Request an inventory

Centralized View of Firmware Versions and Outliers

Anyone keeping a server fleet centrally in view through KVM Fleet has direct console access at hand when something goes wrong, so that individual outliers can be spotted quickly and dealt with in a targeted way. How the same central view can also be used for day-to-day KVM-over-IP access across multiple locations is described in the article on KVM-over-IP at a fleet-wide scale. Why firmware maintenance is at the same time a security topic is shown in the article on BMC Security.

Frequently asked questions

Can firmware updates be performed while the system is running?

Many firmware updates require a server reboot or, at the very least, a brief interruption of the affected component. In redundant systems, a server can be taken offline from the active cluster while others take over the load. For standalone servers without redundancy, a scheduled maintenance window is usually unavoidable.

How can you test a firmware update in advance without putting your entire fleet at risk?

A small staging group with representative hardware is the standard approach. This group receives the update first and runs under observation for a defined period of time before the rest of the fleet follows. It is important that the staging group actually covers the hardware variants currently in use.

What should you do if a firmware update fails?

A prepared rollback plan makes the difference between a brief incident and a prolonged outage. This includes having the previous firmware version on hand, as well as a documented procedure for accessing and resetting the affected server via the BMC, even if the system no longer boots.

The next step

torck also rolls out firmware updates for its own server fleet via KVM Fleet, with a staging group and a rollback plan as described in this article. The team in Maxhütte-Haidhof, Vienna, and Rabat knows the sources of error from its own practice. More on this on the Product Page.

Questions about this post?

Two sentences about your situation are enough. You’ll hear back from someone who builds systems like these themselves.

We'll respond within one business day.torck · Code mit Drehmoment
Florian Blischke
Managing Director at torck · Over 20 years of software development experience
Florian Blischke is the managing director of torck, based in Maxhütte-Haidhof near Regensburg. He has been working in software development for over 20 years. His focus is custom software, data and AI for industry and retail, ranging from the integration of physical processes and IoT to cloud architecture and data- and AI-driven systems. torck works with its own nearshore team in Rabat. At torck, he oversees projects including the Jouvoli energy platform and the KVM Fleet fleet management product. torck develops software at its locations in Maxhütte-Haidhof, Vienna and Rabat and focuses on software that actually works in operation.

Are you facing the same question?

Since 2017, we have been building software for industry and retail from Maxhütte-Haidhof near Regensburg, with teams in Vienna and Rabat. An initial consultation takes 30 minutes and costs nothing. Afterwards you will know whether the project is worth pursuing, even if the answer is no.

More Articles