WeKnora Blog

WeKnora: Tencent's Open-Source Document Understanding and Retrieval Framework

Published: 2024

WeKnora represents a significant advancement in the field of document understanding and retrieval. Developed by Tencent, this open-source framework leverages Large Language Models (LLMs) to provide deep document understanding, semantic retrieval, and context-aware question-answering capabilities.

What Makes WeKnora Special?

WeKnora stands out in the RAG (Retrieval-Augmented Generation) landscape for several key reasons:

• Production-Ready: Unlike many research projects, WeKnora is battle-tested in production environments, serving as the core technology for the WeChat Dialog Open Platform
• Comprehensive Solution: Provides end-to-end capabilities from document parsing to intelligent Q&A, eliminating the need to piece together multiple tools
• Enterprise Features: Built-in multi-tenant support, scalable architecture, and production-grade infrastructure
• Modern Architecture: Built with Go and Vue.js, following modern software engineering best practices

Key Capabilities

WeKnora offers a comprehensive set of features:

• Document Understanding: Advanced parsing for PDF, Word, Markdown, and other formats with automatic structure detection
• Semantic Retrieval: Vector-based search that understands meaning, not just keywords
• Agent Mode: ReACT agents with tool integration for complex problem-solving
• Knowledge Graphs: Transform documents into visual knowledge graphs showing relationships
• Multi-tenant Architecture: Support for multiple organizations with complete data isolation

Real-World Applications

WeKnora is being used in various scenarios:

• Enterprise knowledge management systems
• Customer support chatbots
• Research paper search and analysis
• WeChat ecosystem integration
• FAQ and documentation systems

Understanding RAG: The Technology Behind WeKnora

Published: 2024

Retrieval-Augmented Generation (RAG) is a powerful paradigm that combines the strengths of information retrieval and language generation. WeKnora implements RAG to provide accurate, context-aware answers from your documents.

How RAG Works

RAG follows a two-stage process:

1. Retrieval: When a question is asked, the system searches through the knowledge base to find relevant document sections
2. Generation: The retrieved context is provided to an LLM, which generates a comprehensive answer based on the relevant information

Why RAG Matters

RAG addresses key limitations of LLMs:

• Provides up-to-date information from your documents
• Reduces hallucinations by grounding answers in source material
• Enables domain-specific knowledge without retraining models
• Allows source attribution and transparency

WeKnora's RAG Implementation

WeKnora enhances standard RAG with:

• Advanced semantic retrieval using vector embeddings
• Reranking algorithms for improved precision
• Hybrid search combining semantic and keyword matching
• Agent mode for multi-step reasoning and tool usage

Building Your First Knowledge Base with WeKnora

Published: 2024

Creating a knowledge base with WeKnora is straightforward, thanks to the intuitive Web UI and comprehensive documentation. Here's a step-by-step guide to get you started.

Step 1: Installation

Start by installing WeKnora using Docker Compose:

Step 2: Create Your Account

Access the Web UI at http://localhost and create your account.

Step 3: Create Knowledge Base

Choose between FAQ or Document knowledge base types, depending on your use case.

Step 4: Configure Models

Set up your embedding and LLM models through the Web UI configuration interface.

Step 5: Upload Documents

Upload your documents using drag-and-drop, folder import, or URL import.

Step 6: Start Asking Questions

Once processing is complete, start asking questions and get intelligent answers!

Agent Mode: Taking Q&A to the Next Level

Published: 2024

WeKnora's Agent mode enables ReACT (Reasoning and Acting) agents that can use tools, reason through problems, and provide comprehensive answers through multiple iterations.

What is Agent Mode?

Agent mode transforms WeKnora from a simple Q&A system into an intelligent assistant that can:

• Use multiple tools to gather information
• Reason through complex, multi-step problems
• Reflect on its answers and refine them
• Search the web for real-time information
• Access external tools via MCP protocol

When to Use Agent Mode

Agent mode is ideal for:

• Complex queries requiring multiple information sources
• Questions that need real-time data (e.g., current events)
• Multi-step reasoning problems
• Scenarios requiring tool integration

Example Use Case

WeChat Dialog Open Platform: Zero-Code AI Deployment

Published: 2024

WeKnora serves as the core technology framework for the WeChat Dialog Open Platform, providing a more convenient approach to deploying intelligent Q&A services within the WeChat ecosystem.

Key Benefits

• Zero-Code Deployment: Simply upload knowledge to quickly deploy intelligent Q&A services
• Efficient Question Management: Support for categorized management of high-frequency questions
• WeChat Ecosystem Integration: Seamlessly integrate into WeChat Official Accounts, Mini Programs, and other WeChat scenarios

Use Cases

The WeChat Dialog Open Platform enables businesses to:

• Provide 24/7 customer support through WeChat
• Answer product questions instantly
• Share company information and policies
• Engage with customers through intelligent conversations

Best Practices for Document Preparation

Published: 2024

To get the best results from WeKnora, proper document preparation is essential. Here are some best practices:

Document Structure

• Use clear headings and subheadings
• Break long documents into logical sections
• Use consistent formatting throughout
• Include a table of contents for long documents

Content Quality

• Write clear, concise content
• Use specific terminology consistently
• Include examples where helpful
• Keep information up-to-date

Metadata and Tags

• Add relevant tags to documents
• Include metadata like author, date, category
• Organize documents into logical groups