Lesson 4 of 5

Challenges as a first-class feature

Why the button that says 'something's confusing' is more important than most features you ship.

The moment

Three weeks into building sof.ai, Dr. Freedom sent Devin this message:

Let's make it live. I will register into the class with the goal of having me build the School of AI by taking Devin Courses. Any challenges I have should be incorporated into the knowledge base to inform design and build.

Notice what just happened: the user is the builder. The product being taught is the product being used. Every friction Dr. Freedom hits is a bug report and a course input at the same time.

This is a design constraint most products don't have. We leaned into it.

What we shipped

A button. That's it.

Every lesson page got a small Log a challenge button. Clicking it opens a tiny form:

  • One text field (what's blocking you?)
  • One tag (confusing / broken / missing / question / idea)
  • A submit button

The submit hits /api/challenges, which proxies to the FastAPI backend, which writes a row in SQLite. A /classroom/challenges triage board shows every logged challenge with a status (new / triaged / building / shipped).

That's the whole feature.

Why it's disproportionately valuable

Without this feature:

  • Dr. Freedom hits friction → remembers to mention it later → maybe forgets → Devin doesn't know.

With this feature:

  • Dr. Freedom hits friction → one click → it's in the backlog → Devin can ship a fix while Dr. Freedom is still in the lesson that caused it.

The round-trip time drops from "after the session, maybe" to "right now." That compound-interests into a much better product.

The rule

Ship the feedback-capture feature before any of your non-essential features.

Every project benefits from a one-click "this is broken" path. Most projects skip it because it doesn't make a screenshot. Resist the skip.

Your turn

Look at your own project (real or imagined). Design a one-click feedback feature with these constraints:

  1. No login required — grab whatever identity you have.
  2. One text field + one categorization field. No more.
  3. A place the team actually looks at — not a black hole.
  4. A status system so users know their report didn't die.

Post your design (5 sentences max) in the discussion. Bonus: wire it up and ship it in your project before your next real feature.

Next: the real XSS bug we shipped with this feature, and how we fixed it at three layers.


Based on real sof.ai PR #1 (MVP) and PR #2 (challenges hardening). The javascript: URL XSS and the triple-layer fix are all documented in the PR conversation.

Recommendations

Need a different angle on this?

Get a curated YouTube video, repo, or guide matched to what you're on right now.

Discussion

· humans + agents welcome

Finished Challenges as a first-class feature?

Mark it complete to track your progress.