Claude – AI Capabilities and Limitations

AI Capabilities and Limitations

Human Competencies                                               – AI Properties

Delegation                                                                                – Steerability

Description                                                                              – Working memory

Discernment                                                                          -Next token prediction

Diligence                                                                                    – Knowledge

Generative AI – Creates new content instead of already pre-trained content or classifying or analysing existing content.

Pre-Training – Trained on massive amounts of data to learn patterns

Fine-Tuning – Refined to be broadly safe, ethical  and helpful

Next Token Prediction – Where do AI answers come from?

Knowledge – What does AI actually know?

Working Memory – What is AI paying attention to right now?

Steerability – How much am I in control?

Next token prediction

Capability Zone                                                    Limitation Zone

Summarization

Reformatting                                                           Novel Territory

Common Concepts   ————————-         Obscure topics

Drafting in a familiar style

Practical Implications:

What this Enables                                              Where this fails

Fluent in register                                                   Hallucination

Rapid synthesis                                                    Inconsistency

Strong pattern recognition                                    Misplaced confidence

Coherent continuation

What Human do:

Confident tone is not an accuracy signal

Specificity is where fabrication concentrates

Treat outputs as drafts to verify

Ask where on continuum your task sits

Lean on product surfaces

Model can’t tell grounded from invented

KNOWLEDGE

What AI knows and don’t:

Knows – Its pre trained data

• What generative AI knows comes entirely from training data and is frozen at the knowledge cutoff. Without tools, it has no access to any information after that date.
  
  • Capability zone: topics that appeared frequently, recently (within training), and consistently in training data.
  • Limitation zone: rare, post-cutoff, niche, local, or contested topics.
  • Characteristic failures: staleness, uneven coverage, inherited bias in what counts as “default” or “normal,” and inability to attribute where knowledge came from.
  • Web search, retrieval (RAG/MCPs), and tool use exist specifically to patch these gaps by giving the model access to information it was never trained on.
• 4D connection: Knowledge unevenness is core to Delegation. Understanding where the model is well-stocked versus thin tells you when to hand off, when to supply context yourself, and when to go elsewhere.

Working Memory:

AI Context Window:

Your prompts

AI Responses

Any other Info you have shared

Product features that push the edge out

• Memory: persists facts across sessions
• Compaction / summarization: condenses old turns to free room
• Projects / workspaces: standing docs reliably in context
• Skills: minimize context use until needed
• Larger context windows: push the cliff further out

• Working Memory is the fact that the AI model has a fixed context window that it can attend to.
  
  • Capability zone: your material fits comfortably, the session is current, you’re supplying relevant context.
  • Limitation zone: very long documents or conversations, expecting continuity across sessions, burying critical info in the middle of long input.
  • This property has a cliff rather than a gradient. Silent truncation is the failure mode, and you won’t always be warned.
  • The model doesn’t learn from your corrections. It only responds to what’s currently in context.
  • Memory features, compaction, projects, larger windows, and multi-agent workflows all exist to push this cliff further out.
• 4D connection: Working Memory is what Description acts on. Knowing how the window works tells you how to structure context, when to front-load, and when to start fresh.

Context Degradation – When more context makes things worse

The words you remembered likely cluster at the beginning and end of the list. The middle gets lost. This is the primacy–recency effect — and LLMs show the same bias.

Prompting Fit

The practical advice is straightforward: put your most important instructions at the beginning and end of the context. If a constraint absolutely must be followed, state it early in the system prompt and restate it near the end. Don’t rely on the model to give equal weight to everything in between.

Context degradation is the reason that “just give it more context” is not always the answer. Every piece of context you add pushes other pieces further into the middle — the attention dead zone. This is the core tension of context engineering: not just what to include, but where to put it and what to leave out.

Key takeaway

More context ≠ better results. The model’s attention is finite. Curate ruthlessly, place strategically, and repeat what matters.

Steerability in AI Models:

–              Follow the direction

Two training stages of AI:

Pretraining – Training on massive amounts of data to learn patterns

Fine Tuning – Refined to be broadly safe, ethical and helpful

Product features that push the edge out

• System prompts / custom instructions: standing directions that don’t dilute
• Code execution: offload math to an actual interpreter
• Visible reasoning: catch drift at step two, not the final answer
• Structured output modes: cut down on letter-over-spirit wandering

Key takeaways

• Steerability means the model follows instructions via Next Token Prediction.
  
  • Capability zone: short, concrete, verifiable instructions. Format specs, length limits, explicit roles.
  • Limitation zone: long chains of reasoning, abstract or ambiguous instructions, anything requiring native numerical or logical precision.
  • Characteristic failures: reasoning drift (small errors compound) and letter-over-spirit (the instruction was honored but the intent wasn’t).
  • System prompts, code execution, visible reasoning, and structured output modes exist to keep your intent from diluting.
  • When an instruction is followed literally but uselessly, restate the goal. Repeating the instruction with more force won’t close the gap.
• 4D connection: Steerability is what makes Description powerful and what bounds it. Understanding the gap between words and intent changes how you write prompts and where you insert checkpoints.

·       Every instruction has a gap between letter and spirit. You close it by stating the goal.

·       “Make it shorter” – You meant: surface the ask for a skimmer.

·       “Make it more professional” – You meant: reframe for a different audience and channel.

·       “Add more detail”- You meant: expand only what drives a decision.

Two properties meeting: diagnosing what went wrong:

When Properties Collide – Most real-world AI failures are two properties meeting at the same time.

Next Token Prediction+Steerability:  Confidently wrong reasoning

Next Token Prediction generates fluent, confident-sounding chains of logic. Steerability dutifully follows your complex prompt step by step. But small errors compound — and the confident tone never wavers.

Fix →Use visible reasoning to catch drift early, or offload precise steps to code execution.

Working Memory+Steerability  Long-conversation drift

Your early constraints fade as the conversation grows. Steerability follows whatever instructions are most salient now — so later messages quietly overwrite the earlier ones.

Fix →Re-supply critical context, or start fresh with essentials up front.

Next Token Prediction+Knowledge  Hallucinated citations

The model generates citation-shaped text — plausible titles, real-sounding journals — but there’s a knowledge gap underneath. It can’t tell the difference between what it knows and what it’s fabricating.

Fix →Verify specifics independently, or use source grounding so the model retrieves real documents.

Working Memory+Steerability  Long-conversation drift

Your early constraints fade as the conversation grows. Steerability follows whatever instructions are most salient now — so later messages quietly overwrite the earlier ones.

Fix →Re-supply critical context, or start fresh with essentials up front.

Knowledge+Steerability  Agreeable bad premises

You state something incorrect in your prompt. The model’s Knowledge might “know” better, but Steerability defaults to following your framing — especially if you sound confident.

Fix →Explicitly invite pushback: “Tell me if my assumption is wrong.”

Knowledge+Working Memory  Stale context vs. trained knowledge

Your Working Memory supplies a document that contradicts what Knowledge learned in training. The model may blend both — producing answers that are neither faithful to your source nor to its training.

Fix →Be explicit about which source takes priority: “Use only the attached document” or “Use your training.”

Key takeaways

• Real-world failures are usually two properties interacting, not one.
• Diagnostic pairs to recognize:
  
  • Next Token Prediction + Knowledge (hallucinated specifics)
  • Working Memory + Steerability (long-conversation drift)
• Naming the properties at play points you straight to the fix: verify specifics, re-supply context, offload to code execution, or invite pushback.
• This diagnostic move is Discernment applied. You evaluate better when you know what kind of wrong you’re looking at.

Hallucinated citations: Fabricated citation : Next token prediction meeting knowledge gap

Drift over a long conversation: Working memory fading steerability

Two halves of one system:

1.        Do your quick Internal check

2.        Adjustments:

1.        Verification

2.        Context

3.        Checkpoint