The rapid adoption of lightweight, on-device Large Language Models (Tiny LLMs) has created a critical bottleneck in personalization, knowledge freshness, and reasoning capabilities. While millions of users deploy these models for everyday tasks, their ability to learn, update, and reason remains limited, requiring costly and slow interventions via centralized Big LLM APIs or fine-tuning.

We introduce the Knowledge Cache Graph (KCG) and Cache-Augmented Generation (CAG) — a decentralized, verifiable, and efficient knowledge reasoning framework. Instead of relying on retraining or direct weight manipulation, KCG+CAG enables Distillation on Demand (DoD), where knowledge is distilled, validated, and recorded in a public, immutable knowledge cache. This approach empowers Tiny LLMs to dynamically retrieve, reason over, and consume high-quality, validated knowledge via fast Selective Contextual Reasoning (SCR) pipelines, dramatically reducing inference costs, latency, and vendor lock-in.

By creating an ecosystem where knowledge grows, self-validates, and becomes reusable, KCG+CAG transforms AI reasoning into an open, democratic, and self-improving infrastructure for millions of Tiny LLMs.

The rise of Tiny LLMs — lightweight, on-device large language models — is transforming the AI landscape by bringing generative intelligence to billions of devices. However, this explosion of local inference introduces a fundamental bottleneck in personalization, knowledge freshness, and continuous learning:

• Static and outdated models: Once deployed, Tiny LLMs quickly become stale, as they cannot natively update their knowledge base or integrate new information without retraining.
• Costly and centralized fine-tuning: Existing methods for updating models, such as LoRA or QLoRA fine-tuning, require cloud GPUs, expert intervention, and significant time and money.
• Dependency on Big LLM APIs: Users and apps frequently fall back on Big LLM APIs (GPT, Claude, Gemini) to access fresh or complex knowledge, incurring high costs and introducing privacy, latency, and control issues.
• Absence of a shared, reusable knowledge memory: Tiny LLMs operate in isolation, lacking access to a federated, verified, and continuously growing knowledge layer they can rely on.

These limitations block the scalability of Tiny LLM reasoning capabilities, fragment knowledge across devices, and create inefficiencies both for users and for the broader AI ecosystem.

There is a critical need for a new approach where Tiny LLMs can dynamically acquire, reason over, and integrate fresh, verified knowledge — without retraining, without centralization, and without sacrificing privacy or efficiency.

To overcome the personalization and reasoning bottleneck for Tiny LLMs, we propose the Knowledge Cache Graph (KCG) combined with Cache-Augmented Generation (CAG) — a decentralized knowledge reasoning layer designed for scalable, efficient, and continuous learning without retraining.

KCG is a decentralized, immutable, and verifiable knowledge graph layer, built on top of permanent storage solutions like Arweave or IPFS. It stores:

• Distilled knowledge entries (facts, QA, reasoning chains).
• Verified entity relations and semantic links.
• Embedded key-value caches for fast retrieval.
• Proof-of-knowledge metadata ensuring data integrity and consensus validation.

CAG introduces a Cache-Augmented Generation pipeline where Tiny LLMs no longer rely solely on RAG (retrieval-augmented generation) or direct inference from Big LLMs. Instead:

• Tiny LLMs first query local or Gateway KV caches, pre-filled from the KCG layer.
• Utilize Selective Contextual Reasoning (SCR) pipelines to reason over retrieved knowledge without invoking external APIs.
• Fallback to Distillation on Demand (DoD) requests to Big LLMs only when necessary, ensuring minimal usage of expensive inference services.

DoD allows Tiny LLMs and DoD Agents to trigger on-demand distillation of new knowledge when gaps or outdated data are detected:

• Distilled knowledge is submitted to Gateways.
• Gateways validate, package, and record the knowledge into KCG.
• This ensures that knowledge becomes reusable, validated, and available to all network participants.

• Continuous, lightweight learning for Tiny LLMs without retraining.
• Dramatically reduced inference costs and latency, by leveraging fast local and Gateway caching.
• Decentralized, shared, and verifiable knowledge memory, fostering ecosystem-wide efficiency.
• Open, democratized reasoning layer, removing reliance on centralized AI providers.

This model empowers Tiny LLMs to stay fresh, relevant, and capable — at the edge, in real-time, and with minimal costs.

The KCG+CAG ecosystem is composed of several interconnected layers and actors, forming a decentralized, efficient, and verifiable knowledge reasoning pipeline optimized for Tiny LLMs.

• Immutable Knowledge Layer: Stored on Arweave/IPFS.
• Stores:
  • Distilled knowledge entries (facts, QA pairs, reasoning chains).
  • Semantic relations and ontologies.
  • KV-cache entries for fast retrieval (keyed by embeddings, queries, or topics).
  • Proof-of-knowledge and validator signatures.
• Public and verifiable: Accessible by any Gateway, agent, or user.

• Reasoning Layer on top of KCG and KV-caches.
• Enables Tiny LLMs to perform fast retrieval and reasoning over verified knowledge without retraining.
• Utilizes Selective Contextual Reasoning (SCR) pipelines:
  • Semantic retrieval.
  • Filtering and confirmation.
  • Context-enriched generation.

• Specialized agents or Tiny LLMs acting as initiators of DoD queries.
• Responsible for detecting outdated knowledge, gaps, or novel queries.
• Orchestrate SCR reasoning pipelines.
• Propose distilled knowledge entries to Gateways for validation.

• Federated nodes acting as intermediaries between DoD agents and KCG.
• Responsible for:
  • Knowledge validation and packaging.
  • Recording entries into KCG.
  • Managing off-chain semantic indexes and vector stores for ultra-fast retrieval.
  • Serving enriched knowledge and SCR-ready prompts to Tiny LLMs and agents.
• Gateways control write-access to KCG and enforce quality standards.

• Participate in consensus voting, knowledge verification, and proof-of-knowledge processes.
• Ensure correctness, prevent spam, and provide auditability.
• Optionally contribute to off-chain indexing or SCR pre-computation services.

• On-device Tiny LLM KV-cache: Fast, personalized cache of frequently used knowledge.
• Gateway KV-cache: High-performance, shared caching of validated knowledge and reasoning shortcuts.
• Public KV-layer in KCG: Immutable, community-validated cache ensuring knowledge persistence and transparency.

This modular, multi-layered architecture ensures:

• Tiny LLMs can reason, learn, and update dynamically without retraining.
• Gateways and validators enforce knowledge quality and validation.
• Caching at multiple layers reduces inference costs, latency, and dependence on Big LLMs.

The KCG+CAG system introduces an optimized workflow that allows Tiny LLMs to acquire, reason over, and integrate fresh knowledge without the need for retraining. The process leverages multi-layer caching, Selective Contextual Reasoning (SCR), and Distillation on Demand (DoD) to ensure knowledge freshness, verifiability, and efficiency.

• A Tiny LLM or DoD Agent identifies a knowledge gap, outdated fact, or complex reasoning need.
• It initiates a DoD query, requesting knowledge retrieval, validation, or distillation.

• The DoD Agent or Tiny LLM triggers the Selective Contextual Reasoning (SCR) pipeline via the Gateway:
  • Semantic retrieval from the KCG KV-layer and Gateway off-chain index.
  • Filtering and confirmation step performed locally by the agent or Tiny LLM.
  • Contextual reasoning using enriched prompt with retrieved knowledge.
• If sufficient reasoning is possible using the local or Gateway caches, the process completes without invoking Big LLMs.

• For novel, ambiguous, or high-confidence-required queries, the DoD Agent escalates the request to selected Big LLM APIs (GPT, Claude, Gemini, etc.).
• Multiple models may be queried and compared.

• Based on SCR or Big LLM outputs, the DoD Agent synthesizes a distilled knowledge proposal.
• The proposal includes:
  • Summary.
  • Sources and evidence.
  • Semantic relations and context.
  • Optional embeddings for KV-caching.

• The proposal is submitted to the Gateway.
• The Gateway performs validation steps:
  • Verifies data integrity and formatting.
  • Checks evidence, references, and originality.
  • Optionally invokes Validator consensus for high-value knowledge.
• If validated, the Gateway records the entry into the KCG (Arweave/IPFS).
• A corresponding KV entry is updated in the public KCG KV-layer for efficient future retrieval.

• The DoD Agent receives the finalized KCG TXID as confirmation.
• Validators and Gateways receive incentives for their role.
• A portion of tokens from the DoD query are burned to maintain deflationary tokenomics.

• SCR-first by default reduces calls to Big LLM by up to 80%.
• Multi-layer caching ensures fastest possible retrieval for repeated or similar queries.
• Federated Gateway layer enables scaling and redundancy of knowledge retrieval and validation services.
• Immutable KCG layer ensures global, shared, and verifiable knowledge memory for all participants.

The KCG+CAG ecosystem is supported by distinct roles, each playing a critical part in ensuring the quality, efficiency, and scalability of the knowledge reasoning pipeline.

• Role: Orchestrators of Distillation on Demand (DoD) queries and reasoning pipelines.
• Responsibilities:
  • Identify knowledge gaps or outdated information.
  • Initiate SCR reasoning pipelines.
  • Aggregate outputs from SCR, local caches, and Big LLM APIs.
  • Propose distilled knowledge entries for validation.
  • Use and maintain local KV-caches for fast reasoning.
• Incentives:
  • Earn rewards for initiating valuable DoD queries.
  • Benefit from lower inference costs via SCR and caching.

• Role: Federated nodes serving as intermediaries between agents and the KCG.
• Responsibilities:
  • Validate, package, and record knowledge into KCG.
  • Manage off-chain semantic indexes and vector stores.
  • Operate KV-caching services for Tiny LLMs and agents.
  • Enforce quality and anti-spam measures.
  • Distribute SCR-ready prompts and knowledge to agents.
• Incentives:
  • Earn transaction fees and validation rewards.
  • Maintain economic sustainability by hosting caching and reasoning infrastructure.

• Role: Guardians of knowledge correctness and consensus.
• Responsibilities:
  • Participate in voting and validation of proposed knowledge entries.
  • Ensure factual correctness, source verification, and semantic consistency.
  • Provide audit trails and proof-of-knowledge attestations.
  • Optionally participate in dispute resolution and governance processes.
• Incentives:
  • Earn validation rewards from token flows.
  • Receive staking benefits and governance rights.

• Role: Specialized nodes supporting Gateways.
• Responsibilities:
  • Maintain lightweight off-chain semantic graphs and vector indexes.
  • Serve SCR pipelines with ultra-fast retrieval.
  • Optimize query routing and ranking.
• Incentives:
  • Earn rewards from Gateway operators or network treasury.

The Knowledge Cache Graph (KCG) serves as the immutable, decentralized knowledge memory layer for the ecosystem. It is optimized to store, retrieve, and reason over distilled knowledge efficiently, while ensuring verifiability and traceability.

• Core units of validated knowledge.
• Contain:
  • Distilled summaries.
  • Supporting evidence and sources.
  • Metadata (creation date, proposer, validator signatures).
  • Optional embeddings for KV-layer indexing.

• Discrete concepts, objects, or named entities.
• Serve as graph nodes for semantic linking.
• Example: "Sparse Transformer", "Switch Transformer".

• Semantic links between entities and knowledge entries.
• Types:
  • isA
  • relatedTo
  • usedBy
• Include descriptions, confidence scores, and provenance data.

• Multi-step logical explanations or derivations.
• Stored as ordered steps with supporting evidence.
• Enable Tiny LLMs to shortcut complex reasoning by using pre-validated chains.

• Standardized question-answer pairs.
• Cover frequent queries and enable ultra-fast retrieval for common reasoning needs.

• All data is stored as JSON-LD following linked data principles.
• Ensures compatibility with semantic web standards and decentralized querying.

• Distilled entries and reasoning chains are indexed by:
  • Query embeddings.
  • Canonical questions.
  • Topics and categories.
• This layer supports ultra-fast retrieval by agents and Tiny LLMs.

• Gateways and off-chain nodes maintain semantic graphs of relations and entities.
• These indexes support complex reasoning paths and chaining queries.

• Each entry contains:
  • TXID from Arweave/IPFS (content-addressed, immutable).
  • Validator signatures and consensus proofs.
  • Timestamps and provenance records.

• Efficient retrieval for Tiny LLMs without requiring expensive Big LLM inference.
• Verifiability and immutability via Arweave/IPFS storage.
• Semantic reasoning-ready via graph relations and reasoning chains.
• Optimized for caching and reuse via KV-layer and embeddings.

To maximize reasoning efficiency, minimize inference costs, and enable dynamic knowledge integration for Tiny LLMs, the KCG+CAG architecture incorporates advanced caching strategies enhanced by Selective Contextual Reasoning (SCR) pipelines.

Inspired by state-of-the-art research (arXiv:2503.05212), SCR enables Tiny LLMs to perform lightweight, dynamic reasoning over external knowledge caches without modifying model weights.

• Step 1: Semantic Retrieval — Retrieve relevant knowledge entries from the KV-cache and semantic graph indexes.
• Step 2: Confirmation and Filtering — Tiny LLMs or Gateways filter, confirm, and deduplicate retrieved entries, ensuring contextual fit and factual accuracy.
• Step 3: Contextual Reasoning — Construct an enriched prompt using confirmed knowledge, allowing Tiny LLMs to perform high-quality reasoning locally.
• Step 4: Fallback to DoD and Big LLMs — Only if SCR fails or lacks confidence, a DoD escalation is triggered.

• Location: Directly on user devices.
• Purpose: Fast, personalized retrieval for frequent or user-specific knowledge.
• Latency: Sub-20 ms access time.

• Location: Gateways or local edge servers.
• Purpose: Shared, community-level cache of validated knowledge and reasoning chains.
• Latency: 50–200 ms access time.

• Location: Immutable storage (Arweave/IPFS).
• Purpose: Community-validated, permanent cache of distilled knowledge and reasoning blocks.
• Latency: 300–1000 ms (direct access), faster via Gateway indexes.

• Gateways maintain lightweight off-chain semantic graphs, vector indexes, and retrieval services.
• These indexes accelerate SCR pipelines, enabling sub-second retrieval even from large, decentralized knowledge graphs.
• They also serve as a Federated Knowledge Mesh, ensuring redundancy, locality, and resilience.

• Up to 80% of reasoning queries served locally or via Gateway SCR pipelines, drastically reducing Big LLM API calls.
• Dynamic reasoning capabilities without retraining or fine-tuning.
• Privacy-preserving local reasoning with fallback to decentralized DoD processes.
• Efficient caching and routing optimized for Tiny LLM environments.

By combining SCR with multi-layered caching, KCG+CAG establishes an agile, self-learning reasoning infrastructure, enabling Tiny LLMs to stay fresh, responsive, and efficient at the edge.

The KCG+CAG system can be seamlessly integrated and deployed on top of PEAK Protocol, leveraging its existing decentralized infrastructure, consensus mechanisms, and privacy-enhancing features.

• Validator Network and Governance: KCG can adopt PEAK's validator network, staking mechanisms, and DAO governance, reducing the need to bootstrap a separate validator economy.
• ZK Proof Layer: PEAK's ZK Layer can serve as the foundation for query privacy, access control, and proof-of-knowledge attestations within KCG and DoD requests.
• Optimized Querying and Indexing: By utilizing PEAK's subgraph and indexing services, KCG can enhance its off-chain semantic querying layer, accelerating SCR pipelines without duplicating infrastructure.

Application & Agent Layer
└── Tiny LLMs, User Devices, Enterprise Agents

CAG Reasoning Layer (on top of PEAK)
└── Cache-Augmented Generation, SCR Pipelines, DoD Agents, Gateway Cache & Index

KCG Core Layer (on PEAK)
└── Knowledge Cache Graph, Distilled Knowledge, Relations, KV Layer (on-chain)

PEAK Protocol Layer
└── Validator Network, Consensus, Staking, ZK Proof Layer, Subgraph Indexing

The KCG+CAG ecosystem introduces a deflationary, utility-driven token model designed to reward key participants, sustain decentralized infrastructure, and ensure long-term economic balance.

• Every Distillation on Demand (DoD) request initiated by a DoD Agent incurs a fixed token fee.
• This covers:
  • API calls to Big LLMs (if needed).
  • Storage costs (Arweave, off-chain index updates).
  • Validation and recording into KCG.

• Validators and Gateways are compensated in tokens for:
  • Validating knowledge proposals.
  • Maintaining KV-caches and SCR pipelines.
  • Hosting off-chain indexes and retrieval services.
• Rewards are split proportionally based on workload, reputation, and node performance.

• A percentage of each DoD request fee is burned (removed from circulation).
• This creates a deflationary pressure, balancing the incentive system and aligning with knowledge quality over quantity.

• Incentives are designed to reward useful knowledge contribution, validation, and retrieval, not speculative behavior.
• Over time, as the knowledge cache grows and DoD frequency decreases due to efficient SCR pipelines, the system self-balances toward lower operating costs and higher knowledge utility per token spent.

The integrity, fairness, and sustainability of the KCG+CAG ecosystem are ensured by a decentralized governance model and a distributed network of Validator Nodes.

• Knowledge Validation: Review and approve or reject distilled knowledge proposals submitted via Gateways.
• Consensus Voting: Participate in staking-based or delegated voting mechanisms for validating high-value or disputed entries.
• Proof-of-Knowledge Verification: Sign knowledge entries, providing audit trails and trust guarantees.
• Governance Participation: Influence protocol upgrades, tokenomic adjustments, and dispute resolutions.

• Validators can be:
  • Permissionless nodes operated by the community.
  • Elected by governance mechanisms based on reputation, staking, or delegated trust.
• They run lightweight nodes capable of:
  • Verifying knowledge proposals.
  • Participating in voting rounds.
  • Optionally contributing to off-chain indexing and SCR pre-computation services.

• The protocol may adopt:
  • DAO governance models, where token holders vote on key parameters.
  • Federated validator councils, ensuring fast and efficient decision-making with delegated transparency.

• Protocol upgrades and parameter tuning (e.g., DoD fees, validator rewards, burn rates).
• Validator slashing mechanisms for malicious behavior.
• Dispute resolution between agents, validators, and gateways.
• Treasury management and incentive pool allocations.

• Ensure high knowledge quality and verifiability.
• Prevent spam, misinformation, and abuse.
• Balance openness with integrity.
• Enable community participation while ensuring operational efficiency.

The KCG+CAG ecosystem bridges the gap between heavy, centralized LLM inference and lightweight, efficient Tiny LLMs operating at the edge. By introducing an open, decentralized, and verifiable knowledge graph, coupled with Cache-Augmented Generation (CAG) and Selective Contextual Reasoning (SCR), we enable Tiny LLMs to stay continuously updated, smart, and capable — without costly retraining or vendor lock-in.

This paradigm shift turns reasoning and knowledge augmentation into an open, reusable, and community-driven resource, breaking free from centralized control and enabling AI models to reason dynamically using decentralized knowledge caches.

We envision a world where:

• Tiny LLMs become truly autonomous learners, continuously improving and reasoning at the edge.
• Knowledge becomes a public good, verifiable and accessible to all, stored immutably in the Knowledge Cache Graph (KCG).
• Users, agents, and validators collaborate in a self-reinforcing ecosystem, where knowledge grows organically, costs decrease, and reasoning becomes more reliable, democratic, and sovereign.

By adopting the KCG+CAG architecture, we take a significant step toward democratizing AI reasoning, decentralizing knowledge creation, and empowering users everywhere to control, enhance, and benefit from their own intelligent agents.