Use cases / When production breaks
Errors doubled after a release. Was it the deploy?
The error rate doubles the day after a release, two dashboards point two ways, and someone has to decide whether to roll back.
Jigward ties the errors to one change in the release, reproduces the crash in a test and opens a tested draft fix. The rollback call stays with the team.
The film
Chapters
Read the narration
0:00 Errors doubled: roll back or not?
Errors on the orders page just doubled. Is it yesterday's release, or today's promo traffic? A blind rollback could undo good changes and still not help. I'm Jigward. An admin already connected me to this channel, the repo and read-only production logs. One mention makes the question my Task.
0:23 What shipped, and what is failing
First: what actually shipped? Five pull requests, and only two touch the orders code. Those are my suspects. Right now: a few timeouts, and a type error in the order totals, always on orders with a coupon.
0:43 Before and after the release
That error never shows up before the release; it starts six minutes after. The promo didn't cause it, it just brought more coupon orders.
0:55 The changed line, reproduced in a test
There's the link. Anna's change rewrote the exact line in the stack trace. Coupon orders don't have that field. Before proposing anything, I'll prove it. A coupon order in a test fails exactly like production.
1:11 A tested fix as a draft pull request
One guard for lines without a promo. Now the new test passes, and so does the suite. The fix goes up as a draft pull request, with both options: revert Anna's change, or merge this.
1:29 The rollback call stays with the team
It's the release, but only one change in it, so nobody has to roll back everything. I haven't deployed or reverted anything; that call stays with the team.
What changes
Today. Someone lists what shipped and scrolls the commits, someone else searches the logs and compares graphs, and a third person prepares a rollback just in case. Dashboards and error trackers show that errors went up. They do not tie the new error to the line a change rewrote, reproduce it, or put a revert and a fix side by side, so the call is made on a hunch.
-
Before: Rolling back the whole release on a hunch, and losing four unrelated changes with it.
With Jigward: One pull request named as the cause, with the log lines and the changed line that show it.
-
Before: Two dashboards and two theories in the incident channel.
With Jigward: Both explained. The promo did not cause the error; it sent more coupon orders into it.
-
Before: A fix written in a hurry after the rollback.
With Jigward: A test that fails the way production does, and a draft pull request that makes it pass.
What Jigward does
@Jigward errors on /orders doubled this afternoon. Was it yesterday's release, and do we roll back?
Connected beforehand
- The incident channel is named for the storefront's Context.
- The storefront repository is connected through GitHub.
- An AWS account is connected, so Jigward can read the production log groups an admin allowed.
-
Lists what shipped
It compares the two release tags. Five pull requests went out, and two of them touch the orders code.
-
Finds the new error
A log search over the last three hours shows a few timeouts and a type error that hits only orders with a coupon.
-
Compares the releases
A two-day Logs Insights query by release shows the type error never appeared before the release and started six minutes after it, while timeouts stayed flat.
-
Ties it to a line of code
The stack trace points at the exact line one pull request rewrote, which reads a field that coupon orders do not have.
-
Reproduces it, then fixes it
A new test with a coupon order fails the way production does. One guard makes it pass, and the whole suite passes too.
-
Answers in the thread
It opens a draft pull request with both options, revert the one change or merge the fix, and answers in the incident thread with what it did not check.
Jigward in this case
-
Slack
The question arrives as a mention in the incident channel, and the answer goes back to its thread.
-
Tasks
The mention becomes a Task that keeps every log search, the diff and the test runs.
-
AWS CloudWatch Logs
A log search finds the new error, and a Logs Insights query compares it across the two releases.
-
GitHub
Jigward reads what the release changed and opens the fix as a draft pull request. It never merges.
-
Workspace
The failing test and the full suite run on Jigward's own copy of the repository.
What comes back, and what stays with the team
What comes back
- GitHub A draft pull request with the cause, the new test, the fix and the choice between reverting one change and merging the fix.
- The incident thread in Slack The answer, with links to both pull requests and the Task, and what was not checked.
What the team decides
- Whether to revert one pull request, merge the fix or roll back the whole release.
- Every deploy. Jigward deploys and reverts nothing.
What Jigward did not check
- Checkout and payment flows.
- Other services and their log groups.
- Whether other order types reach the same code.
Meet your next engineering coworker
Bring a question
from your own project.
See how Jigward can help your team understand the system and move the next task forward.
Jigward is in closed testing. Accounts are created by hand; sending a request does not create one.