====== Graph Prompting ======

Graph prompting integrates graph structures -- such as knowledge graphs, relational networks, and graph neural network (GNN) embeddings -- into LLM prompts to enhance the model's ability to reason over structured, relational data. This addresses a key limitation of LLMs: their difficulty in precisely handling factual and relational information encoded in graph form.((Tian et al. 2023, [[https://arxiv.org/abs/2309.15427|Graph Neural Prompting with Large Language Models]]))

===== Approaches =====

==== Text-Based Graph Encoding ====

Graphs are serialized into textual representations that LLMs can process directly. Google Research's "Talk Like a Graph" explores encoding strategies including node ordering, edge notation formats, and subgraph selection methods.((Google Research, [[https://research.google/blog/talk-like-a-graph-encoding-graphs-for-large-language-models/|Talk like a Graph]]))  Key findings:

  * Larger LLMs perform better on graph reasoning tasks due to their capacity for complex patterns.
  * LLMs struggle with specific graph tasks like cycle detection compared to simple algorithmic baselines.
  * Encoding format significantly impacts performance.

==== Graph Neural Prompting (GNP) ====

Graph Neural Prompting combines GNNs with LLMs through a multi-step process:((Tian et al. 2023, [[https://arxiv.org/abs/2309.15427|Graph Neural Prompting with Large Language Models]]))

  - **Subgraph retrieval**: Relevant subgraphs are retrieved from a knowledge graph based on query entities.
  - **GNN encoding**: A graph neural network encodes the subgraph nodes into embeddings.
  - **Cross-modality pooling**: Relevant nodes are selected through cross-attention with the text query.
  - **Domain projection**: A projector aligns graph embeddings with the LLM's text embedding space.
  - **Prompt construction**: The resulting graph neural prompt (a soft prompt) is prepended to text embeddings.

This produces instance-specific prompts per query, unlike dataset-level methods like standard prompt tuning.

==== In-Context Learning with Graphs (GraphICL) ====

GraphICL uses structured prompt templates to capture graph structure in Text-Attributed Graphs, enabling in-context learning without training. It outperforms specialized graph LLMs in resource-constrained settings.((GraphICL, [[https://aclanthology.org/2025.findings-naacl.131/|ACL Findings 2025]]))

===== Knowledge Graph Integration =====

Knowledge graphs provide factual and structural knowledge to augment LLMs through retrieval-augmented mechanisms:

  * Subgraphs are retrieved based on query entities from questions and answer options.
  * Graph-derived information is encoded into prompts via soft prompts or textual serialization.
  * This plug-and-play integration avoids retraining LLMs.

Integration approaches fall into four categories:
  * **GNNs as prefixes**: Graph encodings prepended to LLM input.
  * **LLMs as prefixes**: LLM features used to enhance graph processing.
  * **Full integration**: Joint training of GNN and LLM components.
  * **LLMs only**: Graph information serialized as text for pure LLM processing.

===== Benchmark Results =====

Graph Neural Prompting achieved significant improvements:((Tian et al. 2023, experimental results))

  * **+13.5% accuracy** over baselines (e.g., prompt tuning) on frozen LLMs across six commonsense and biomedical datasets.
  * **+1.8% accuracy** improvement when LLMs are additionally fine-tuned.
  * Outperformed both standard prompt tuning and LoRA on multiple benchmarks.
  * Ablation studies confirmed the contribution of each component (GNN, pooling, projector).

GraphICL outperformed specialized graph LLMs and GNNs on out-of-domain text-attributed graph benchmarks.

===== Practical Applications =====

  * **Commonsense reasoning**: Grounding LLM responses in knowledge graph facts.
  * **Biomedical question answering**: Leveraging medical knowledge graphs for clinical QA.
  * **Node classification**: Classifying entities in social or citation networks.
  * **Link prediction**: Predicting missing relationships in knowledge graphs.
  * **Graph reasoning**: Tasks like cycle detection, edge existence checking, and shortest path finding.

===== Limitations =====

  * **Graph quality dependency**: Performance relies on the coverage and accuracy of the underlying knowledge graph.
  * **Alignment challenge**: Bridging graph and text embedding spaces requires careful projection.
  * **Scalability**: Large knowledge graphs increase retrieval and encoding overhead.
  * **Task-specific tuning**: GNP components may need retraining for different domains or graph types.

===== See Also =====

  * [[prompt_engineering]]
  * [[generate_knowledge_prompting]]
  * [[chain_of_thought_prompting]]
  * [[few_shot_prompting]]
  * [[program_aided_language_models]]

===== References =====