Practise AI in Engineering Teams one question at a time. Answer out loud or in writing, get graded, and see exactly what you missed.
What does it mean for a team to be "AI-native" rather than simply using AI tools?
An AI-native team designs its delivery system around safe, repeatable AI collaboration. Simply using AI means adding tools to an otherwise unchanged process.
| Dimension | Uses AI tools | AI-native |
|---|---|---|
| Workflow | Individuals prompt as needed | Specifications, context, and tests guide agents |
| Control | Existing review absorbs output | Permissions and risk gates limit actions |
| Success | Tool adoption or generated code | Delivery quality, effort, and defects |
People still own product choices, architecture, and released changes. The team maintains reusable instructions and evaluations, keeps project knowledge retrievable, and strengthens verification as generation becomes faster.
Missed:
“Mainly because it looks finished. What the model writes is tidy and confident, so it's tempting to skim it, but it doesn't know our system. It'll happily call a method that doesn't exist in the library version we're on, or handle the happy path and quietly break a business rule nobody told it about. So I read the whole diff against the ticket, look up any call I don't recognise, and try the edge cases myself before I approve it. My name's on the PR either way, so I treat it like code I wrote.”
Knew itThey have it: they name why fluent AI output invites false trust, check unfamiliar calls, diff against the ticket, and test edges themselves. Would be stronger with a word on checking security or auth handling specifically.
Missed: Doesn't mention checking authorization, secrets handling, or external calls specifically
Missed: Doesn't mention that generated tests need independent review since they can repeat the same mistake as the implementation
Get your own answers graded
Sign up free. Your account comes with 50 credits for grading, and it remembers what you missed.
What it covers
- AI-native workflow foundations
- Reviewing AI-generated code
- Testing AI-written code
- Debugging with AI
- Documentation with AI
- Large-scale changes and migrations
- Security, licensing and governance
- Review capacity and verification at scale
- Engineering metrics with AI
- Architecture and long-term codebase health
- Skills, learning and hiring
All 49 questions
AI-native workflow foundations
What makes a team AI-native, vibe coding versus disciplined delegation, spec-first work and treating prompts and agent configs as engineering artifacts.
- What does it mean for a team to be "AI-native" rather than simply using AI tools?
- What is the "prompt-and-pray" anti-pattern, and what does a healthier workflow look like?
- What is vibe coding, and when is it acceptable on an engineering team? Pro
- What is spec-first development, and why has it become more prominent in AI-assisted engineering? Pro
- Should prompts, agent configurations, and evaluation harnesses be treated as engineering artifacts? What does that mean in practice? Pro
Reviewing AI-generated code
Spotting plausible-but-wrong code, hallucinated APIs, over-engineering, off-idiom code and slop PRs, and setting how far to trust each change.
- Why should AI-generated code be reviewed at least as carefully as human-written code?
- What is a hallucinated API, and how do you catch one before it causes damage?
- What is an "AI slop" pull request, and why do maintainers consider it harmful? Pro
- What does "plausible-but-wrong" code look like in practice, and why do experienced reviewers miss it? Pro
- Why do AI models tend to over-engineer, and how do you counter it? Pro
- How do you evaluate whether AI-generated code is consistent with your codebase's idioms, and why does it matter? Pro
- How would you build and maintain a taxonomy of AI-typical defects for your review process? Pro
- How do you calibrate how much to trust AI output for a given task, rather than applying uniform skepticism? Pro
Testing AI-written code
Using TDD as guardrails for agents, the pitfalls of generated tests, the test oracle problem, and property-based and mutation testing as checks.
- How can test-driven development act as guardrails when you delegate coding to an AI agent?
- What are the most common pitfalls of letting an AI generate your test suite?
- How does property-based thinking improve testing of AI-generated code? Pro
- What is mutation testing, and why is it especially valuable for auditing AI-written test suites? Pro
- Explain the test oracle problem in the context of AI-generated tests, and how to mitigate it. Pro
Debugging with AI
Using a model as a talking rubber duck, for log and stack-trace analysis, and in hypothesis-driven debugging where evidence decides.
Documentation with AI
The benefits and risks of AI-written docs, writing docs that agents consume, and keeping cheap-to-generate docs from going stale.
Large-scale changes and migrations
Choosing codemods or agents for large changes, verifying migrations in batches, and making huge refactor diffs reviewable.
Security, licensing and governance
Security risks typical of AI code, licensing and IP, disclosure norms, usage policy, vendor selection and rolling AI out across an organization.
- Should engineers disclose when a change was substantially AI-generated? What are the emerging norms? Pro
- What security issues are characteristic of AI-generated code, and how should security review adapt? Pro
- What should a company's AI usage policy for engineering actually cover? Pro
- What licensing and IP risks come with AI-generated code, and how do teams manage them? Pro
- How would you evaluate and select an AI coding tool or vendor for an engineering organization? Pro
- Design an organization-wide rollout of AI engineering practices for a 500-engineer company that balances velocity gains against risk. Pro
Review capacity and verification at scale
Adapting review norms to higher PR volume, risk-tiered review gates, the verification bottleneck and a lifecycle built around cheap generation.
- Pull request volume doubled after your team adopted coding agents. How should review norms change? Pro
- Design a review gate system for AI-generated changes in an organization shipping hundreds of PRs a week. Pro
- When agents can generate far more code than humans can review, how do you manage the verification bottleneck? Pro
- Generation is now cheap and verification is the bottleneck. How should the software development lifecycle be re-architected around that inversion? Pro
Engineering metrics with AI
Why velocity metrics mislead once AI writes code, which outcome metrics still hold up, and how to design metrics that resist Goodhart's law.
- Why do traditional velocity metrics become misleading once AI generates much of the code? Pro
- Which engineering metrics remain meaningful in an AI-heavy workflow, and how do you instrument them honestly? Pro
- How do you design an engineering metrics system that resists Goodhart's law specifically in an AI-amplified environment? Pro
Architecture and long-term codebase health
Which architecture decisions humans must own, and controlling tech debt, duplication and churn to keep a codebase understandable over years.
- AI assistants are linked to rising code duplication and churn. How do you measure and control tech-debt accumulation? Pro
- In an AI-heavy organization, which architecture decisions must humans own, and how do you operationalize that boundary? Pro
- Under sustained high AI churn, how do you preserve long-term codebase health and human comprehension over multiple years? Pro
Skills, learning and hiring
Learning with AI without shallow knowledge, avoiding skill atrophy, growing juniors, choosing skills to delegate, and hiring and interviewing when AI is in use.
- What is your strategy for learning an unfamiliar stack or codebase with AI without ending up with shallow knowledge? Pro
- What do interviewers actually probe when they ask about your AI-assisted workflow, and what distinguishes a strong answer? Pro
- How does an experienced engineer avoid skill atrophy while working heavily with AI? Pro
- How should teams onboard and develop junior engineers when AI handles most traditional junior tasks? Pro
- AI can now complete most take-home assignments and coding screens. How would you redesign engineering hiring to get real signal? Pro
- Looking ahead, which engineering skills should teams still develop deeply versus deliberately delegate to AI, and how do you decide? Pro
All questions
Showing of 49What does it mean for a team to be "AI-native" rather than simply using AI tools?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
An AI-native team designs its delivery system around safe, repeatable AI collaboration. Simply using AI means adding tools to an otherwise unchanged process.
| Dimension | Uses AI tools | AI-native |
|---|---|---|
| Workflow | Individuals prompt as needed | Specifications, context, and tests guide agents |
| Control | Existing review absorbs output | Permissions and risk gates limit actions |
| Success | Tool adoption or generated code | Delivery quality, effort, and defects |
People still own product choices, architecture, and released changes. The team maintains reusable instructions and evaluations, keeps project knowledge retrievable, and strengthens verification as generation becomes faster.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
Why should AI-generated code be reviewed at least as carefully as human-written code?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
AI-generated code deserves at least normal scrutiny because fluent syntax can hide false assumptions, invented interfaces, and insecure patterns. It can also widen the change beyond what was asked.
- Read the complete diff and compare its behavior with the requirement.
- Verify unfamiliar APIs and dependencies against authoritative sources for the version you use.
- Test boundary cases and failures. Then check authorization, the handling of secrets and other data, and every external call.
- Review generated tests independently, because they may repeat the implementation's mistake.
The submitter owns every line. Set review depth from the change's impact and reversibility, not from how quickly or confidently the code appeared.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
What is the "prompt-and-pray" anti-pattern, and what does a healthier workflow look like?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
Prompt-and-pray means sending a vague request, accepting a large result, and hoping it works. It skips controlling the scope and gathering evidence.
A healthier workflow looks like this:
- Define the goal and the acceptance criteria. Add the relevant context, constraints, and non-goals.
- Ask for a plan when the work is nontrivial, then correct faulty assumptions.
- Make one small, reviewable batch and inspect its complete diff.
- Run focused checks before continuing, and require approval for risky commands.
- Refine the specification or context when results are wrong.
The assistant drafts and explains work. The engineer controls direction, permissions, and the evidence required for acceptance.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
How can test-driven development act as guardrails when you delegate coding to an AI agent?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
Test-driven development gives an agent an executable contract and a short feedback loop, making delegated work narrower and easier to inspect.
- Review a test that expresses one required behavior and confirm it fails for the right reason.
- Ask for the smallest implementation that makes the test pass.
- Protect tests from unapproved edits so the agent cannot weaken the contract.
- Add boundaries, invalid inputs, and failure paths, then rerun focused checks.
- Inspect the implementation and repeat with the next behavior.
A green suite proves only the specified cases, so human review and broader verification still matter.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
What are the most common pitfalls of letting an AI generate your test suite?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
Generated tests often mirror the implementation instead of the requirement, so a polished suite and high coverage can still provide weak protection.
- Shared mistake: Code and assertions can encode the same wrong interpretation.
- Weak signal: Tautologies, excessive mocks, and snapshots may never detect realistic faults.
- Missing risk: Boundaries, invalid input, permissions, and failure paths are easy to omit.
- False confidence: Tests may pass even after the behavior they claim to protect is broken.
Start expected outcomes from an independent specification. Review assertions, add integration coverage at important boundaries, and use mutation testing where its cost is justified.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
What is a hallucinated API, and how do you catch one before it causes damage?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
A hallucinated API is an invented method, option, or endpoint whose plausible name makes it look real. Models invent whole packages the same way.
- Let the compiler, type checker, or language server flag unknown symbols.
- In dynamic code, run a focused experiment instead of trusting an explanation.
- Check unfamiliar calls in the official documentation for the version the project installs.
- Review a package's registry entry, publisher, and source before installing it. Invented names are a supply-chain risk, because an attacker can publish a real package under one.
- Keep new dependencies behind human approval.
When sources conflict, trust the executable project and authoritative documentation over the model's confidence.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
How does rubber-duck debugging change when the duck can talk back?
Answer it yourself first - out loud, or typed below.
How should your speech become text?
Listening… your words appear above as you speak - tap Stop when you're done.
Recording · cr - tap Stop & transcribe when you're done.
Transcribing with AI…
Voice:
Last attempt -
A talking rubber duck can question your assumptions, propose competing hypotheses, and suggest experiments. A silent duck only forces you to explain the bug aloud.
Start with observed facts, expected behavior, and one testable hypothesis. Ask the assistant what evidence would distinguish that idea from the alternatives, then run the smallest useful experiment.
Guard against anchoring on a fluent story. Share only relevant, redacted logs, and verify claims in code, documentation, or runtime output. The assistant widens the search for causes, but direct observation decides which explanation survives.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
What is an "AI slop" pull request, and why do maintainers consider it harmful?
Should engineers disclose when a change was substantially AI-generated? What are the emerging norms?
What are the benefits and risks of using AI to generate documentation?
What is vibe coding, and when is it acceptable on an engineering team?
Pull request volume doubled after your team adopted coding agents. How should review norms change?
What does "plausible-but-wrong" code look like in practice, and why do experienced reviewers miss it?
Why do AI models tend to over-engineer, and how do you counter it?
How does property-based thinking improve testing of AI-generated code?
What is mutation testing, and why is it especially valuable for auditing AI-written test suites?
How do you use an AI assistant effectively for log and stack-trace analysis?
Describe hypothesis-driven debugging with a model in the loop.
When should a large-scale change be done with a codemod versus an agent, and how do the two combine?
How do you structure incremental verification for an AI-assisted migration across a large codebase?
What does it mean to write documentation for AI consumption, and what belongs in it?
AI makes docs cheap to produce but they still go stale. How do you keep documentation fresh?
How do you evaluate whether AI-generated code is consistent with your codebase's idioms, and why does it matter?
What security issues are characteristic of AI-generated code, and how should security review adapt?
What should a company's AI usage policy for engineering actually cover?
What licensing and IP risks come with AI-generated code, and how do teams manage them?
How would you evaluate and select an AI coding tool or vendor for an engineering organization?
Why do traditional velocity metrics become misleading once AI generates much of the code?
What is spec-first development, and why has it become more prominent in AI-assisted engineering?
What is your strategy for learning an unfamiliar stack or codebase with AI without ending up with shallow knowledge?
What do interviewers actually probe when they ask about your AI-assisted workflow, and what distinguishes a strong answer?
Design a review gate system for AI-generated changes in an organization shipping hundreds of PRs a week.
AI assistants are linked to rising code duplication and churn. How do you measure and control tech-debt accumulation?
Which engineering metrics remain meaningful in an AI-heavy workflow, and how do you instrument them honestly?
How does an experienced engineer avoid skill atrophy while working heavily with AI?
How should teams onboard and develop junior engineers when AI handles most traditional junior tasks?
Explain the test oracle problem in the context of AI-generated tests, and how to mitigate it.
In an AI-heavy organization, which architecture decisions must humans own, and how do you operationalize that boundary?
When agents can generate far more code than humans can review, how do you manage the verification bottleneck?
What techniques make a very large AI-generated refactor diff actually reviewable?
How would you build and maintain a taxonomy of AI-typical defects for your review process?
Should prompts, agent configurations, and evaluation harnesses be treated as engineering artifacts? What does that mean in practice?
How do you calibrate how much to trust AI output for a given task, rather than applying uniform skepticism?
Design an organization-wide rollout of AI engineering practices for a 500-engineer company that balances velocity gains against risk.
Generation is now cheap and verification is the bottleneck. How should the software development lifecycle be re-architected around that inversion?
Under sustained high AI churn, how do you preserve long-term codebase health and human comprehension over multiple years?
AI can now complete most take-home assignments and coding screens. How would you redesign engineering hiring to get real signal?
How do you design an engineering metrics system that resists Goodhart's law specifically in an AI-amplified environment?
Looking ahead, which engineering skills should teams still develop deeply versus deliberately delegate to AI, and how do you decide?
This answer is part of Pro.
The full written answer, with the trade-offs and follow-ups an interviewer will probe.
No matches
Try a different filter or search term.
AI in Engineering Teams in pictures.
A quick recap of AI in Engineering Teams in 38 short animated pictures, one idea at a time. About 10 min to get the whole topic back in your head before you walk in. Pause any picture, slow it down, or step through it.
Pro The first chapter is free to play. The other 34 pictures come with Pro.
AI in Engineering Teams cheatsheet
- AI-native workflow01
- Review AI code02
- Review at team scale03
- Tests as guardrails04
- Debug with a model05
- Migrations & refactors06
- Docs & agent config07
- Policy, vendors & IP08
- Metrics & tech debt09
- Skills, juniors & hiring10
- The model11
- + 5 more inside
This picture is part of Pro. The first chapter is free to play; the other 34 pictures come with Pro.
Get Pro · $10/mo Already Pro? Sign in42 of 49 AI in Engineering Teams answers are in Pro.
Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.
- Full answers + code
- AI explanations, simpler or deeper
- 1,000 AI credits / month
- Cancel anytime
Change topic
Pick a different technology or stack. Your current topic stays put until you choose a new one.
MEAN
MongoDB, Express, Angular, Node.jsMERN
MongoDB, Express, React, Node.jsDjango
Python Full-Stack DevelopmentRuby on Rails
Convention over ConfigurationServerless on AWS
Serverless Architecture on AWSInterviewers also test these - they're common to every stack, whichever one you picked above.
Flutter Mobile
Flutter Cross-Platform Mobile DevelopmentInterviewers also test these - they're common to every stack, whichever one you picked above.
Spring Boot
Enterprise Java Development.NET
Microsoft EcosystemVue
Vue.js, Vite, TypeScript, Tailwind, Node.jsGo Backend
Golang, gRPC, PostgreSQL, Redis, RabbitMQInterviewers also test these - they're common to every stack, whichever one you picked above.
FastAPI
Python, FastAPI, SQLAlchemy, PostgreSQLReact Native
React, TypeScript, Redux, FirebaseiOS Native
Swift, SwiftUI, UIKit, FirebaseAndroid Native
Java, Jetpack Compose, FirebaseDevOps / Platform
Docker, Kubernetes, Terraform, CI/CDInterviewers also test these - they're common to every stack, whichever one you picked above.
AI Engineer
LLMs, RAG, Agents, EvalsAI-Powered Developer
Claude Code, Copilot, Agentic WorkflowsCore SWE Interview Prep
Data structures, algorithms, OS, concurrency, networking, gitInterviewers also test these - they're common to every stack, whichever one you picked above.
Interviewers also test these - they're common to every stack, whichever one you picked above.