Skip to content

Fix various commands to work with Redis Cluster #338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
May 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 1 addition & 24 deletions .github/workflows/claude.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,27 +36,4 @@ jobs:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}

# Define which tools Claude can use
allowed_tools: |
Bash(git status)
Bash(git log)
Bash(git show)
Bash(git blame)
Bash(git reflog)
Bash(git stash list)
Bash(git ls-files)
Bash(git branch)
Bash(git tag)
Bash(git diff)
Bash(make:*)
Bash(pytest:*)
Bash(cd:*)
Bash(ls:*)
Bash(make)
Bash(make:*)
View
GlobTool
GrepTool
BatchTool

# Your Anthropic API key (stored as a GitHub secret)
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
allowed_tools: "Bash(git status),Bash(git log),Bash(git show),Bash(git blame),Bash(git reflog),Bash(git stash list),Bash(git ls-files),Bash(git branch),Bash(git tag),Bash(git diff),Bash(make:*),Bash(pytest:*),Bash(cd:*),Bash(ls:*),Bash(make),Bash(make:*),View,GlobTool,GrepTool,BatchTool"
21 changes: 15 additions & 6 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
make test-all

test:
name: Python ${{ matrix.python-version }} - ${{ matrix.connection }} [redis ${{ matrix.redis-version }}]
name: Python ${{ matrix.python-version }} - ${{ matrix.connection }} - redis-py ${{ matrix.redis-py-version }} [redis ${{ matrix.redis-version }}]
runs-on: ubuntu-latest
needs: service-tests
env:
Expand All @@ -89,7 +89,8 @@ jobs:
# 3.11 tests are run in the service-tests job
python-version: ["3.9", "3.10", 3.12, 3.13]
connection: ["hiredis", "plain"]
redis-version: ["6.2.6-v9", "latest", "8.0-M03"]
redis-py-version: ["5.x", "6.x"]
redis-version: ["6.2.6-v9", "latest", "8.0.1"]

steps:
- name: Check out repository
Expand All @@ -116,14 +117,22 @@ jobs:
run: |
poetry install --all-extras

- name: Install specific redis-py version
run: |
if [[ "${{ matrix.redis-py-version }}" == "5.x" ]]; then
poetry add "redis>=5.0.0,<6.0.0"
else
poetry add "redis>=6.0.0,<7.0.0"
fi

- name: Install hiredis if needed
if: matrix.connection == 'hiredis'
run: |
poetry add hiredis

- name: Set Redis image name
run: |
if [[ "${{ matrix.redis-version }}" == "8.0-M03" ]]; then
if [[ "${{ matrix.redis-version }}" == "8.0.1" ]]; then
echo "REDIS_IMAGE=redis:${{ matrix.redis-version }}" >> $GITHUB_ENV
else
echo "REDIS_IMAGE=redis/redis-stack-server:${{ matrix.redis-version }}" >> $GITHUB_ENV
Expand All @@ -135,7 +144,7 @@ jobs:
credentials_json: ${{ secrets.GOOGLE_CREDENTIALS }}

- name: Run tests
if: matrix.connection == 'plain' && matrix.redis-version == 'latest'
if: matrix.connection == 'plain' && matrix.redis-py-version == '6.x' && matrix.redis-version == 'latest'
env:
HF_HOME: ${{ github.workspace }}/hf_cache
GCP_LOCATION: ${{ secrets.GCP_LOCATION }}
Expand All @@ -144,12 +153,12 @@ jobs:
make test

- name: Run tests (alternate)
if: matrix.connection != 'plain' || matrix.redis-version != 'latest'
if: matrix.connection != 'plain' || matrix.redis-py-version != '6.x' || matrix.redis-version != 'latest'
run: |
make test

- name: Run notebooks
if: matrix.connection == 'plain' && matrix.redis-version == 'latest'
if: matrix.connection == 'plain' && matrix.redis-py-version == '6.x' && matrix.redis-version == 'latest'
env:
HF_HOME: ${{ github.workspace }}/hf_cache
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
Expand Down
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,17 @@ pyrightconfig.json
[Ll]ocal
pyvenv.cfg
pip-selfcheck.json
env
venv
.venv

libs/redis/docs/.Trash*
.python-version
.idea/*
.vscode/settings.json
.python-version
tests/data
.git
.cursor
.junie
.undodir
116 changes: 116 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# CLAUDE.md - RedisVL Project Context

## Frequently Used Commands

```bash
# Development workflow
make install # Install dependencies
make format # Format code (black + isort)
make check-types # Run mypy type checking
make lint # Run all linting (format + types)
make test # Run tests (no external APIs)
make test-all # Run all tests (includes API tests)
make check # Full check (lint + test)

# Redis setup
make redis-start # Start Redis Stack container
make redis-stop # Stop Redis Stack container

# Documentation
make docs-build # Build documentation
make docs-serve # Serve docs locally
```

Pre-commit hooks are also configured, which you should
run before you commit:
```bash
pre-commit run --all-files
```

## Important Architectural Patterns

### Async/Sync Dual Interfaces
- Most core classes have both sync and async versions (e.g., `SearchIndex` / `AsyncSearchIndex`)
- Follow existing patterns when adding new functionality

### Schema-Driven Design
```python
# Index schemas define structure
schema = IndexSchema.from_yaml("schema.yaml")
index = SearchIndex(schema, redis_url="redis://localhost:6379")
```

## Critical Rules

### Do Not Modify
- **CRITICAL**: Do not change this line unless explicitly asked:
```python
token.strip().strip(",").replace(""", "").replace(""", "").lower()
```

### README.md Maintenance
**IMPORTANT**: DO NOT modify README.md unless explicitly requested.

**If you need to document something, use these alternatives:**
- Development info → CONTRIBUTING.md
- API details → docs/ directory
- Examples → docs/examples/
- Project memory (explicit preferences, directives, etc.) → CLAUDE.md

## Testing Notes
RedisVL uses `pytest` with `testcontainers` for testing.

- `make test` - unit tests only (no external APIs)
- `make test-all` - includes integration tests requiring API keys

## Project Structure

```
redisvl/
├── cli/ # Command-line interface (rvl command)
├── extensions/ # AI extensions (cache, memory, routing)
│ ├── cache/ # Semantic caching for LLMs
│ ├── llmcache/ # LLM-specific caching
│ ├── message_history/ # Chat history management
│ ├── router/ # Semantic routing
│ └── session_manager/ # Session management
├── index/ # SearchIndex classes (sync/async)
├── query/ # Query builders (Vector, Range, Filter, Count)
├── redis/ # Redis client utilities
├── schema/ # Index schema definitions
└── utils/ # Utilities (vectorizers, rerankers, optimization)
├── optimize/ # Threshold optimization
├── rerank/ # Result reranking
└── vectorize/ # Embedding providers integration
```

## Core Components

### 1. Index Management
- `SearchIndex` / `AsyncSearchIndex` - Main interface for Redis vector indices
- `IndexSchema` - Define index structure with fields (text, tags, vectors, etc.)
- Support for JSON and Hash storage types

### 2. Query System
- `VectorQuery` - Semantic similarity search
- `RangeQuery` - Vector search within distance range
- `FilterQuery` - Metadata filtering and full-text search
- `CountQuery` - Count matching records
- Etc.

### 3. AI Extensions
- `SemanticCache` - LLM response caching with semantic similarity
- `EmbeddingsCache` - Cache for vector embeddings
- `MessageHistory` - Chat history with recency/relevancy retrieval
- `SemanticRouter` - Route queries to topics/intents

### 4. Vectorizers (Optional Dependencies)
- OpenAI, Azure OpenAI, Cohere, HuggingFace, Mistral, VoyageAI
- Custom vectorizer support
- Batch processing capabilities

## Documentation
- Main docs: https://docs.redisvl.com
- Built with Sphinx from `docs/` directory
- Includes API reference and user guides
- Example notebooks in documentation `docs/user_guide/...`
50 changes: 19 additions & 31 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ numpy = [
{ version = ">=1.26.0,<3", python = ">=3.12" },
]
pyyaml = ">=5.4,<7.0"
redis = "^5.0"
redis = ">=5.0,<7.0"
pydantic = "^2"
tenacity = ">=8.2.2"
ml-dtypes = ">=0.4.0,<1.0.0"
Expand Down Expand Up @@ -68,8 +68,8 @@ pytest-xdist = {extras = ["psutil"], version = "^3.6.1"}
pre-commit = "^4.1.0"
mypy = "1.9.0"
nbval = "^0.11.0"
types-redis = "*"
types-pyyaml = "*"
types-pyopenssl = "*"
testcontainers = "^4.3.1"
cryptography = { version = ">=44.0.1", markers = "python_version > '3.9.1'" }

Expand Down
Loading