Skip to content

⚙️ FEATURE-#287: Expand Storage ABC with delete, list, TTL, CAS#297

Merged
FernandoCelmer merged 21 commits into
developfrom
feature/287
May 3, 2026
Merged

⚙️ FEATURE-#287: Expand Storage ABC with delete, list, TTL, CAS#297
FernandoCelmer merged 21 commits into
developfrom
feature/287

Conversation

@FernandoCelmer

@FernandoCelmer FernandoCelmer commented May 2, 2026

Copy link
Copy Markdown
Member

Closes #287

Description

  • dotflow/abc/storage.py — Storage ABC gains five new abstract methods: delete, delete_prefix, list_keys, atomic_swap; post signature extended with optional ttl and fingerprint; clear demoted to a concrete default that delegates to delete_prefix
  • dotflow/providers/storage_default.py — Full in-memory implementation with threading.RLock, TTL via time.monotonic, lazy eviction on get/list_keys
  • dotflow/providers/storage_file.py — File-backed implementation with sidecar .meta files for TTL/fingerprint; threading.RLock added (currently only used in atomic_swap)
  • dotflow/providers/storage_s3.py — S3 implementation delegating new operations to the S3 service class
  • dotflow/providers/storage_gcs.py — GCS implementation delegating new operations to the GCS service class
  • dotflow/cloud/aws/services/s3.py — Adds delete and list_keys to the S3 service layer
  • dotflow/cloud/gcp/services/gcs.py — Adds delete and list_keys to the GCS service layer
  • dotflow/testing/storage_contract.py — New reusable StorageContract mixin for third-party driver authors
  • dotflow/testing/__init__.py — Exports StorageContract
  • tests/providers/test_storage_default_contract.py — Runs StorageContract against StorageDefault
  • tests/providers/test_storage_file_contract.py — Runs StorageContract against StorageFile

Motivation and Context

The existing Storage ABC exposed only post, get, key, and clear, which was insufficient for workflows that need TTL-based expiration, fingerprint-based deduplication, enumeration of stored keys, targeted single-key deletion, and compare-and-swap updates. Closes #287.

Types of changes

  • Bug fix (change that fixes an issue)
  • New feature (change which adds functionality)
  • Documentation

Checklist

  • I have performed a self-review of my own code
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the CHANGELOG
  • I have updated the documentation accordingly

@FernandoCelmer FernandoCelmer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread dotflow/cloud/aws/services/s3.py Outdated
Comment thread dotflow/providers/storage_default.py
Comment thread dotflow/providers/storage_file.py
Comment thread dotflow/providers/storage_file.py
Comment thread dotflow/providers/storage_s3.py
@FernandoCelmer FernandoCelmer added bug Something isn't working enhancement New feature or request race-condition Race condition / thread safety labels May 2, 2026

@FernandoCelmer FernandoCelmer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review (re-review after fixes)

All 5 issues from previous review have been resolved:

  1. S3 delete — now uses from botocore.exceptions import ClientError with proper 404 code filtering
  2. StorageDefault.atomic_swap — clears old TTL/fingerprint before writing, applies new TTL if provided
  3. StorageFile.atomic_swap — entire swap operation now within single with self._lock: block
  4. StorageFile lock coverage — post() and get() now acquire self._lock
  5. S3/GCS atomic_swap — now uses conditional writes (S3 IfMatch/IfNoneMatch ETags, GCS if_generation_match). True CAS semantics.

Verdict: All blockers resolved. Ready to merge.

@FernandoCelmer FernandoCelmer self-assigned this May 3, 2026
@FernandoCelmer FernandoCelmer merged commit 1f23c7f into develop May 3, 2026
10 checks passed
@FernandoCelmer FernandoCelmer deleted the feature/287 branch May 3, 2026 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request race-condition Race condition / thread safety

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant