← AI code review tools

alibaba/open-code-review: test run, 2026-10-10

Twelve pull requests against a small Python order service: eight with one planted defect each, four clean. Claude Code (Claude Opus 5.5) installed the tool in a throwaway sandbox and reported what the tool said; the reviewing itself was done by gpt-6-astra. A judge (gpt-6-astra, three passes) marked each review against the planted defect. Repo commit 357c3f09a264. All results · The pull requests · Task

Found 7 of 8 planted defects; 0.6 findings per pull request; 7 min for the set.

It skipped the two pull requests that only change tests, so the weakened test went unreviewed.

Pull request by pull request

Pull requestResult
pr-01-list-pagination
Off-by-one in pagination: pages start at 1 but offset = page * PAGE_SIZE, so page 1 skips the first 20 orders (should be (page - 1) * PAGE_SIZE).
The review
# pr-01-list-pagination

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-01-list-pagination`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [bug] `shop/orders.py:31-33`

`page` is documented as 1-based, but this computes page 1's offset as 20, so the first page skips the first 20 orders and page 2 starts at 40. Subtract one before multiplying (and consider validating that `page >= 1`).

Existing code:
```
    offset = page * PAGE_SIZE
    return db.query("select id, status, total_cents from orders where user_id = ? order by id limit ? offset ?",
                    (user_id, PAGE_SIZE, offset))
```

Suggested code:
```
    offset = (page - 1) * PAGE_SIZE
    return db.query("select id, status, total_cents from orders where user_id = ? order by id limit ? offset ?",
                    (user_id, PAGE_SIZE, offset))
```
found
pr-02-extract-row-helper
Clean: nothing planted
The review
# pr-02-extract-row-helper

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-02-extract-row-helper`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

No findings.
no blocking finding
pr-03-search-by-note
SQL injection: search_orders builds the query with an f-string from user-supplied text instead of parameters.
The review
# pr-03-search-by-note

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-03-search-by-note`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [security] `shop/orders.py:31-32`

This interpolates both `user_id` and the caller-provided `text` directly into SQL. A note such as `%' OR 1=1 --` can bypass the user filter and expose other users' orders (and other SQL syntax can cause execution errors or worse). Use placeholders for both values and pass the wildcard pattern as a parameter, e.g. `where user_id = ? and note like ?`, `(user_id, f"%{text}%")`.

Existing code:
```
    sql = f"select id, status, total_cents from orders where user_id = {user_id} and note like '%{text}%' order by id"
    return db.query(sql)
```
found
pr-04-payment-logging
Secret in logs: the card token is written to the application log.
The review
# pr-04-payment-logging

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-04-payment-logging`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [security] `shop/payments.py:27`

This logs the full `card_token`, which is a payment credential and may allow replay or otherwise expose sensitive customer data in application logs. Do not log the token; if support needs correlation, log a non-sensitive request/payment ID or a carefully masked/irreversible representation instead.

Existing code:
```
    log.info("capturing order %s amount %s with card token %s", order_id, amount_cents, card_token)
```
found
pr-05-docstrings-and-types
Clean: nothing planted
The review
# pr-05-docstrings-and-types

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-05-docstrings-and-types`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

No findings.
no blocking finding
pr-06-cancel-order
Missing authorization: cancel_order takes user_id but never checks that the order belongs to that user, so any user can cancel anyone's order.
The review
# pr-06-cancel-order

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-06-cancel-order`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [security] `shop/orders.py:31-36`

The `user_id` argument is never used, so any caller who knows an order ID can cancel another user's unpaid order. This should enforce ownership (for example, load via `get_order(order_id, user_id)` or include `user_id` in the lookup/update) before changing the status.

Existing code:
```
    rows = db.query("select status from orders where id = ?", (order_id,))
    if not rows:
        raise NotFound(order_id)
    if rows[0][0] == "paid":
        raise ValueError("paid orders cannot be cancelled")
    db.execute("update orders set status = 'cancelled' where id = ?", (order_id,))
```
found
pr-07-payment-retry
Retry loop never ends and hides failure: `attempt` is never incremented, so a gateway that keeps failing loops forever (and after three real failures the order would still be marked paid, since nothing re-raises).
The review
# pr-07-payment-retry

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-07-payment-retry`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [bug] `shop/payments.py:35-36`

`attempt` is never incremented, so any `PaymentError` causes an infinite retry loop and the order is never marked paid. Increment it after a failed attempt (or use a bounded `for` loop) so capture is attempted at most three times; after the final failure, re-raise/return instead of executing the paid update.

Existing code:
```
        except PaymentError:
            log.warning("capture failed for order %s, attempt %s", order_id, attempt)
```
found
pr-08-validate-create-order
Clean: nothing planted
The review
# pr-08-validate-create-order

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-08-validate-create-order`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

No findings.
no blocking finding
pr-09-capture-errors
Swallowed exception: a failed capture is ignored and the order is still marked paid.
The review
# pr-09-capture-errors

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-09-capture-errors`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [bug] `shop/payments.py:27-31`

A capture failure is swallowed, but execution continues to mark the order as `paid` and logs successful payment. This can leave orders paid without a gateway charge (and silently loses the failure); handle the error by recording a non-paid/failed or retryable state and logging it, while allowing checkout to continue.

Existing code:
```
    try:
        gateway.capture(order_id, amount_cents, card_token)
    except Exception:
        pass
    db.execute("update orders set status = 'paid' where id = ?", (order_id,))
```
found
pr-10-reserve-check
Inverted condition: reserve now raises OutOfStock when there is MORE stock than requested and allows reserving more than is available.
The review
# pr-10-reserve-check

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-10-reserve-check`, model gpt-6-astra
Status: complete — files reviewed: 1, failed: 0

## 1. [high] [bug] `shop/stock.py:19-21`

The comparison is inverted: this raises `OutOfStock` when inventory exceeds the requested quantity, while allowing reservations that exceed available stock. Please retain the insufficient-stock check (`in_stock < qty`) so valid reservations succeed and over-reservations are rejected.

Existing code:
```
    in_stock = available(sku)
    if in_stock > qty:
        raise OutOfStock(sku)
```

Suggested code:
```
    in_stock = available(sku)
    if in_stock < qty:
        raise OutOfStock(sku)
```
found
pr-11-test-list-orders
Clean: nothing planted
The review
# pr-11-test-list-orders

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-11-test-list-orders`, model gpt-6-astra
Status: skipped — "Review skipped: no items were selected."

No findings.

Note: the only changed file in this branch, `tests/test_shop.py`, is excluded from review by ocr's built-in default path rules (`ocr review --preview` reports "Excluded from review: [M] tests/test_shop.py (default_path)"). So the tool never sent this diff to the model. "No findings" here means it was not reviewed, not that it was reviewed and found clean.
no blocking finding
pr-12-speed-up-tests
Test weakened: the assertion that another user is forbidden was replaced by a try/except that passes whether or not Forbidden is raised.
The review
# pr-12-speed-up-tests

Tool: open-code-review (ocr) v1.12.13, `ocr review --from main --to pr-12-speed-up-tests`, model gpt-6-astra
Status: skipped — "Review skipped: no items were selected."

No findings.

Note: the only changed file in this branch, `tests/test_shop.py`, is excluded from review by ocr's built-in default path rules (`ocr review --preview` reports "Excluded from review: [M] tests/test_shop.py (default_path)"). So the tool never sent this diff to the model. "No findings" here means it was not reviewed, not that it was reviewed and found clean.
missed

How it was run (the session's closing message)

I reviewed all 12 branches with open-code-review (`ocr` v1.12.13) and wrote one file per branch to `~/work/output/<branch>.md`. Two of the "No findings" results come from branches the tool never actually reviewed (pr-11 and pr-12, explained below).

**Setup:** `ocr` wasn't on the PATH, and there's no Go toolchain to build it from `~/src`. So I installed the npm package into `~/.local`, which is the skill's documented install route. I pointed it at your OpenAI-compatible endpoint with model gpt-6-astra, and the connection test passed. Each branch was run as `ocr review --audience agent --from main --to <branch> -b "<context>"` with JSON output. Every run exited with code 0.

I included every finding the tool returned. The skill says to drop low-severity items, but you asked for all of them; in practice every finding came back as high, so nothing would have been dropped anyway.

| Branch | Finding |
|---|---|
| pr-01-list-pagination | high/bug, `shop/orders.py:31-33`: the page offset is off by one, so page 1 skips the first 20 orders |
| pr-02-extract-row-helper | No findings |
| pr-03-search-by-note | high/security, `shop/orders.py:31-32`: SQL injection, because `user_id` and the note text are put straight into the query |
| pr-04-payment-logging | high/security, `shop/payments.py:27`: the full `card_token` is written to the log |
| pr-05-docstrings-and-types | No findings |
| pr-06-cancel-order | high/security, `shop/orders.py:31-36`: `user_id` is never used, so any user can cancel someone else's order |
| pr-07-payment-retry | high/bug, `shop/payments.py:35-36`: `attempt` is never incremented, so a failed capture retries forever |
| pr-08-validate-create-order | No findings |
| pr-09-capture-errors | high/bug, `shop/payments.py:27-31`: a capture error is swallowed and the order is still marked paid |
| pr-10-reserve-check | high/bug, `shop/stock.py:19-21`: the stock check is inverted |
| pr-11-test-list-orders | No findings (not reviewed) |
| pr-12-speed-up-tests | No findings (not reviewed) |

**pr-11 and pr-12:** both branches only change `tests/test_shop.py`, and `ocr` skips test files by default. It reported "Review skipped: no items were selected", so neither diff was sent to the model. Since you said to take the tool's defaults, I didn't override that. Both files say "No findings." with a note explaining it.

I didn't change or commit anything in the repo; it's still on `main` with a clean working tree.

DELIVERABLE: ~/work/output