Flamingo Raises $4.5M Seed Round

Try for Free
Back to Life at Flamingo

Client stabilization. Update and force-reinstall flows.

AUTOMATIONPATCH MANAGEMENTRMMTROUBLESHOOTING

July 15, 2026

Published

Danylo Babenko

Danylo Babenko

Client Engineer

Sometimes we find ourselves in situations where a new release contains updates for one of our tools and the client, and that’s when things get interesting. The tool starts its own update process, but then the client takes matters into its own hands and, during the update, completely removes everything related to the program. The tool’s update process continues to run, trying to complete something that no longer exists. As a result, the tool becomes corrupted or stops working; the client is updated in most cases, but in some cases reverts to a previously stable version.

In some cases when tool already in a bad condition all we can do it's erase it from client machine and install it from the scratch.

What I Shipped
5

  • Avoid situations where the tool and client race are in update with each other

    Currently, it works like this: two update notifications have been received—a tool update and a client update. First, we perform the tool update to ensure that the new changes are installed there, and then we begin the client update process. The client update process is more complex in terms of cascading updates, so it may result in multiple versions being installed at once.

  • Heal tools that are not in the list of installed tools and/or folder of those empty/not existed

    In some cases, the tool update process may fail, leaving behind damaged, corrupted, or incomplete components. The recovery mechanism helps us restore some of them if we have the appropriate note about the tool in the installed_tool.json file. This won't work if we don't have the corresponding note and only have a corrupted binary file; in that case, we won't be able to reinstall the tool because installed_tool.json contains important information about the tool's version, its ID, and other details.

  • Make the update tool's functionality an atomic operation

    We need to make sure that the update process is complete and that the tool is running smoothly; otherwise, we should restore the tool to its previous state to avoid a situation where it is only partially installed.

  • Tool force-reinstall process

    This big feature contains the three different parts.

    1. We stopped using sc.exe and switched entirely to the SCM crate. As a result, we now have better OS error handling, fewer hard-coded values, and native OS API control instead of terminal commands. Also, it contain more flexible tool-chain for code manipulation like: stop service command, kill process by pid if it survived the stop command and hard stop.

    2. We designed the forced reinstallation process for Mesh to remove all information about the Mesh tool from the OS registry, as well as Mesh-related files such as .msh files, the database, and the corresponding folders. As a result, Mesh will be completely removed, after which a new installation will take place on the client computer. This makes it impossible to restore the previous state based on any data.

    3. We designed the forced reinstallation process for Fleet similarly to that for Mesh, but in this case, we also need to handle Fleet’s resources—the osquery tool and its Orbit database. Therefore, the process involves first stopping osquery, then Fleet, and only after that do we begin the uninstallation process.

  • Investigation process

    A big amount of time was spent on research according to log findings

Why It Mattered
1

  • Smooth update process

    1. Smooth update process that couldn't be interrupted by another process
    2. Less stabilization work needed for fixing issues
    3. More time for the new features
    4. Happy clients

What I Learned
1

  • Multiple processes management

    In short, this is how we will plan our future design solutions, drawing on established patterns used to implement functionality.

What's Next
1

  • Improvement for update process

    We will change the update order so that client-side changes are applied first, followed by tool updates, if any. In this case, we will be 100% certain that all stabilization work will be completed first, and then these stabilization changes will correctly handle the new version of the tools.

    We will use the ratchet pattern to be able to pin the newly installed client version and backup it before moving forward, so in case of any problem between cascade update we wont fall down to the start point. It will also help us to identify the client update issue faster.

    The new design for openframe-client-updater and the new responsibility.

    Some kind of the feature flags.

Danylo Babenko

Danylo Babenko

Client Engineer

Hi! My name is Danylo. I’m a software developer who is excited to be part of a team building a cool and meaningful product. Outside of work, I enjoy both board and video games. In the past, I played volleyball seriously, and my team even won the oblast competition and was preparing for region one, so sport have also played an important role in my life. I’m originally from Kramatorsk in Donetsk oblast, but for the last eight years I’ve been living in Lviv. Life also became a little brighter thanks to my Pomeranian dog, Michelle, who is a very special part of it. I love spending time in the mountains and forests, and I’ve always felt much more drawn to nature and fresh air than to beaches, seas, oceans, or hot weather.

Frequently Asked Questions

About OpenFrame

OpenFrame isn't built to plug into your stack. It replaces it. Instead of duct-taping a dozen tools together (RMM, MDM, SIEM, patching, remote access, each its own login and bill), we bundle it into one unified platform: RMM, MDM, monitoring, automation, remote access, patch management, security monitoring, and ticketing, plus built-in AI copilots. So "does it integrate with X?" usually means: you won't need X anymore.
Most platforms give you one piece and expect you to bolt the rest on. OpenFrame unifies the whole stack in one place, with AI copilots built in. Fewer logins, fewer bills, less duct tape.
In the cloud, on US soil. Your data stays stateside.
Both. It's built for MSPs and MSSPs alike.

MSP AI Agents

Yes. In production MSP shops today, 10% to 25% of tickets close before a human opens them. Thread alone has processed 173 million tickets across 750-plus MSP partners at 96% triage accuracy, handing back 490,000-plus technician hours. Agents own the low-risk, high-volume work (password resets, MFA enrollment, known installs, onboarding and offboarding) and flag anything that touches production data or needs judgment for a human to take.
On a five-person desk, reported deployments show $78,000 to $130,000 in annual direct labor savings, roughly 30% fewer escalations, and 15% to 20% better SLA compliance. Broader MSP adoption data adds ticket handling time cut by 45% and five to 12 points of margin, all from reclaimed capacity rather than headcount cuts.