Add index empty and missing support

[tylerhutcherson]

PR implements support to index missing fields as well as empty fields.

[Copilot]

Pull Request Overview

Adds support for indexing documents with missing or empty fields by extending schema attributes, serialization, field conversion, and query filtering.

• Introduces index_missing and index_empty attributes across field definitions and ensures they propagate to Redis field creation.
• Updates serialization logic to preserve default attribute values.
• Adds is_missing() filter method and comprehensive unit/integration tests for missing/empty field behavior.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/unit/test_schema.py	New tests for missing/empty indexing in IndexSchema.
tests/unit/test_filter.py	Tests for the new is_missing() filter combinations.
tests/unit/test_fields.py	Tests for index_missing/index_empty on all fields.
tests/integration/test_query.py	End-to-end tests covering missing/empty queries.
redisvl/utils/utils.py	Serialization updated to exclude_defaults=False.
redisvl/schema/fields.py	Added attributes and updated as_redis_field methods.
redisvl/query/filter.py	Added is_missing() to FilterField.

Comments suppressed due to low confidence (2)

redisvl/schema/fields.py:201

• [nitpick] The as_redis_field implementations in TextField, TagField, NumericField, and GeoField repeat a common pattern for building kwargs. Consider extracting shared logic into a helper or base method to reduce duplication.

def as_redis_field(self) -> RedisField:

tests/unit/test_fields.py:349

• RedisVectorField is used here but not imported. Add the appropriate import (e.g. from redis.commands.search.field import VectorField as RedisVectorField) at the top of the test file.

assert isinstance(redis_field, RedisVectorField)

[tylerhutcherson]

Need to filter out the tests for the older redis version... Do we still need to test/support that older pre 7.x version ;) @claude can you please review how we filter tests for the older versions of Redis elsewhere and then address that with our new tests we added here?

[abrookins]

@claude where you at

[claude]

Claude encountered an error —— View job

---

I'm analyzing the Redis version filtering patterns and will address the test filtering for the new index empty/missing tests.

Todo List

• Read CLAUDE.md for repo guidelines
• Examine existing Redis version filtering patterns in tests
• Review new tests added in this PR
• Identify which new tests need version filtering
• Apply appropriate filtering to new tests
• Commit changes if needed

Found Redis Version Filtering Patterns

The codebase has established patterns in tests/conftest.py:

• skip_if_redis_version_below(client, min_version, message) for sync tests
• skip_if_redis_version_below_async(client, min_version, message) for async tests
• Common threshold: 7.4.0 for INDEXMISSING/INDEXEMPTY features

Tests Needing Version Filtering

Integration Tests (✅ Already Fixed)

tests/integration/test_query.py - 6 tests already have proper filtering at line 965-970

Unit Tests (❌ Missing Filtering)

Need to add version filtering to 8 unit test functions:

tests/unit/test_fields.py:

• test_field_attributes_index_missing_and_empty() (line 223)
• test_default_index_missing_and_empty_values() (line 271)
• test_redis_field_creation_with_index_attributes() (line 313)
• test_vector_fields_redis_creation_with_index_missing() (line 335)
• test_vector_field_data_includes_index_missing() (line 362)
• test_field_factory_with_new_attributes() (line 391)

tests/unit/test_schema.py:

• test_schema_with_index_missing_and_empty_attributes() (line 183)
• test_schema_serialization_with_new_attributes() (line 250)

[Copilot]

Pull Request Overview

Adds support for indexing missing and empty fields, enhances serialization to preserve new attributes, and centralizes Redis version/module checks in test utilities.

• Introduces index_missing and index_empty in field attribute models and propagates them through as_redis_field and parsing logic.
• Adds is_missing() filter method to all filter types and updates compare_versions handling of new boolean attributes.
• Provides skip_if_redis_version_below and skip_if_module_version_error helpers and refactors tests to use them; updates Docker Compose image default.

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/unit/test_schema.py	New tests for index_missing/index_empty support in schemas
tests/unit/test_filter.py	Added is_missing() method tests for all filter types
tests/conftest.py	Added version/module-skipping utilities for tests
redisvl/schema/fields.py	Extended field attributes and updated Redis field conversion
redisvl/query/filter.py	Added is_missing() to FilterField
redisvl/redis/connection.py	Updated compare_versions signature and parsing of boolean attrs
redisvl/utils/utils.py	Enhanced model_to_dict and shutdown wrapper error handling
tests/docker-compose.yml	Changed Redis image default

Comments suppressed due to low confidence (3)

tests/docker-compose.yml:1

• The Docker Compose file no longer specifies a version header, which may lead to compatibility issues. Re-add version: "3.9" as the first line to declare the Compose file version.

version: "3.9"

tests/integration/test_message_history.py:588

• [nitpick] Avoid using a bare except: which can catch unintended exceptions. Catch RedisModuleVersionError explicitly to prevent masking other errors.

except:

redisvl/utils/utils.py:179

• The code references sys but there is no import for the sys module in this file, which will cause a NameError. Add import sys at the top of the module.

if sys is None or getattr(sys, "_getframe", None) is None:

[abrookins]

This will be super useful. 👍 LGTM!