Ryker / Docs / Alerts and incidents

Alerts and incidents

Ryker can pick up alerts from the tools already posting to Slack. It checks the cause, works on a fix, and reports whether the system recovered. Your team steps in for decisions and actions that require approval.

Connect the systems it needs

Give the alert channel an environment containing the relevant repositories and an Emisar account. In Emisar, allow the read actions needed for investigation and set an approval policy for changes.

In the Slack channel’s Customize flow, choose Be proactive and investigate app alerts in their thread. This lets an actionable app alert start work without a mention. For manual investigations, mentions-only mode is enough.

Start with an alert

An alert should identify the service, environment, time, and symptom, with a link to the logs or metrics. For example, a checkout error-rate alert gives Ryker a starting point for checking recent deployments, reading application logs, and tracing failing requests in the code.

To ask directly, reply in the alert’s thread:

@Ryker investigate these checkout errors in staging. Find the cause, prepare a fix, and check recovery afterward. Bring any required approval back to us here.

Ryker compares the code and intended configuration with current evidence. It reports whether the alert is confirmed, likely, disproved, or still unverified, and explains the impact. A confirmed or likely problem includes a proposed immediate mitigation and lasting fix.

Add missing context in the thread while it works. Ryker asks if it needs a decision or access it does not have. A Terraform review needs the actual plan, supplied in the conversation or retrieved through a read-only tool.

Approve a change through Emisar

When a fix needs an infrastructure action, Ryker requests it through Emisar. Actions allowed by policy can run immediately. Actions needing approval produce an Emisar review card with the action, target, and an Open approval link.

Open the request in Emisar and review the exact action. An authorized person approves or denies it there. Ryker cannot approve its own request, and a Slack reply does not count as Emisar approval.

The card updates as the action progresses. The wait survives a restart and holds no model or worker. Ryker continues in the same conversation when the result arrives.

If the lasting fix belongs in code, Ryker can offer a task. Confirm Start task so it can prepare and test the change in co:op, then open a draft pull request. Your team merges and releases it through its usual process.

Check that the fix worked

After an infrastructure action, Ryker checks its intended effect against fresh read-only evidence where the tools support it. For the checkout example, that could mean comparing the new error rate with the original alert. The command’s result and the service’s recovery are recorded separately.

If recovery takes time, Ryker can wait and check again. For a code fix, verification follows your deployment; a matching deployment event can continue the request. If recovery is still unverified, Ryker says what evidence is missing.

Move coordinated work into an incident room

An incident room is a dedicated Slack channel for an investigation. A configured operator can press Create incident room on Ryker’s offer. The offer also has Investigate for keeping the same work in the original thread.

Channel settings can create a room automatically for a credible, unresolved app alert. A person’s message always needs the operator’s confirmation. Rooms use the source conversation’s environment and invite the configured operators plus any additional people selected for that channel.

Rooms are private by default and named <prefix>-MMDD-<title>-<id>, with inc as the default prefix. The default limit is 25 open rooms per workspace.

The pinned card updates in place with findings, questions, and available controls. Its menu opens the timeline, evidence, handoff summary, and a postmortem draft built from recorded work. You can generate the draft before or after the incident closes.

Archiving a room pauses its investigation until it is unarchived. Deleting it closes the room; Ryker reports this in the alert thread and sends any reply it still owed there. Deleting a room does not discard its code changes.

Review the investigation

Open Activity in the console for the original message, context sent to the model, tool results, and checks on its answer. The evidence records show what supports or contradicts each finding.

To handle the next event in a specific way, save a rule. For regular checks even when no alert arrives, create a schedule.