Crackme obfuscation tags: display, search, submission & review (#17, #162)#163
Open
xusheng6 wants to merge 19 commits into
Open
Crackme obfuscation tags: display, search, submission & review (#17, #162)#163xusheng6 wants to merge 19 commits into
xusheng6 wants to merge 19 commits into
Conversation
Introduces a controlled vocabulary of 21 anti-analysis / obfuscation tags (anti-debugging, string encryption, packers, ...) seeded from the AI-labeled crackmes-RE dataset, and wires them through the whole crackme lifecycle. - app/services/tags.py: 21-class vocabulary, normalize_tags(), dataset URL - Crackmes carry a `tags` field: authors pick tags at submission, tags show on the crackme page (each links to a tag search) with a "?" explaining the tags are AI-generated from write-ups/comments and may be inaccurate - Search: filter by tags (matches crackmes carrying all selected tags) - Reviewers can override tags when reviewing a pending crackme and when editing an approved one - Tag change requests: any logged-in user can request tag adds/removes; a new reviewer queue (dashboard link + count) approves (applies) or rejects them, notifying the requester - FAQ #tags entry; script/import_tags.py to backfill tags from the dataset - Tests for vocabulary, model, submission, search, requests, reviewer overrides Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the tag vocabulary with the dataset's finer sub-labels: specific
anti-debug methods (IsDebuggerPresent, ptrace, ...), packer names (UPX, FSG,
...), and control-flow techniques (CFF, ...), nested under their parent class.
- app/services/tags.py: SUBLABELS, TAG_GROUPS (grouped view), ALL_TAGS (flat
canonical order: each class followed by its sub-labels); normalize_tags now
validates and orders across classes + sub-labels
- Submission, request modal, and reviewer forms render sub-labels indented
under their class; search uses <optgroup> so a category ("any") or a specific
technique can be selected
- import_tags.py also ingests antidebug_methods / packers / controlflow_methods
- Tests for sub-label vocabulary, ordering, grouped structure, submission, and
search-by-sub-label
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Tag chips rendered one-per-line because each chip was followed by a hidden <form> inside a <p>; a <form> start tag auto-closes an open <p>, breaking the inline flow. Use one shared hidden search form and a <div> container so chips sit inline. - The "AI-generated, may be inaccurate" note is no longer always shown; it is revealed by clicking the "?" and now clarifies that new crackmes are tagged by their authors while only older imported tags are AI-generated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Search now runs off URL query parameters so results can be bookmarked and shared (issue #162): - /search accepts GET: a bare /search shows the empty form, any parameter (e.g. /search?tags=Packer) runs the search. POST is kept for backward compatibility; both share one _render_search(source) helper. - The search form and all pagination controls submit via GET (CSRF hidden inputs dropped from these read-only forms so URLs stay clean). - Tag chips on a crackme page are now plain GET links to /search?tags=<url-encoded tag>, so clicking one produces a bookmarkable URL and the tag stays selected in the form on arrival. - Tests for GET-based tag search, the empty state, and chip link encoding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Tag change requests now use a two-list transfer UI: left = not applied, right = applied; click a tag to move it. Moving a technique also applies its category; removing a category removes its techniques. The form submits the full desired "applied" set and the server diffs it against the crackme's current tags to derive add/remove. - On the submission form (and reviewer set-tags / edit forms), ticking a specific technique auto-ticks its parent category, and un-ticking a category clears its techniques, so tags never end up with an orphaned sub-label. - Tags remain optional (a crackme with no matching technique may have none), but the submission text no longer calls them "optional" and asks authors to tag every technique that applies. - Tests updated for the applied-set request semantics and the no-tags path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the click-to-move dual list with an explicit two-list transfer widget: select one or more tags, then move them with the "=>" / "<=" buttons. Adds a Reset (restore original set) and Undo (revert last move) button. Moving a technique into "applied" pulls in its parent category; moving a category out drops its techniques, so a sub-label is never applied without its parent. - New static/js/tag_transfer.js (TagTransfer.init) + shared CSS in custom.css, used in both the tag-change request modal (submits the desired "applied" set) and the reviewer edit-crackme page (submits the full "tags" set). The reviewer edit tags editor moves from grouped checkboxes to this widget. - Test updated: edit page now renders the widget seeded with current tags. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Undo/Reset never worked because the buttons sat outside the element the widget scans (root.querySelector couldn't find them, so no listeners were attached). Wrap the two-list transfer and its action buttons in a single root container on both the request modal and the reviewer edit page. - Enlarge the tag change dialog (modal-lg, wider container, taller lists, bigger click targets) so tags are easier to click. - The "Request a tag change" link now lives inside the "?" help box, beneath "you can request a change", instead of always showing in the header. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- In the tag transfer widget, removing a technique now also removes its parent category when no other technique of that category remains applied (the inverse of "adding a technique adds its category"). - Also fix two NUL bytes that had crept into the Reset comparison line (which made the JS file classify as binary); the comparison now uses JSON.stringify and the file is clean ASCII. Replace the raw arrow glyph with ↳ for the same reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the tags help box, logged-out users now see a "Log in to request a tag change" link instead of a dead-end mention, matching the logged-in link. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dataset now labels named commercial protectors under the packers field (VMProtect, ASProtect, .NET Reactor, ConfuserEx, Enigma, Themida, ExeCryptor, WinLicense, SmartAssembly, PELock, CodeVirtualizer, Dotfuscator) and adds an "Anti-attach / thread suspension" anti-debug method. Add these to the controlled vocabulary (ordered by frequency) so the reimport keeps them instead of dropping them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…el sets) (#17) The dataset taxonomy was updated: obfuscation classes dropped from 21 to 16 (Timing checks, Exception-based, Commercial protector, Stripped/no symbols and Anti-attach/thread tricks were folded into other classes), and three new sub-label families were added: - antidisasm_methods -> Anti-disassembly - crypto_methods -> Crypto / hash algorithm - encryption_methods -> String / data encryption AES / Base64 / RC4 / TEA-XTEA appear under both crypto and encryption, so they are qualified per source ("AES (crypto)" vs "AES (encryption)") to keep each tag under exactly one parent (dataset_sublabel_tag). The import script now ingests all six sub-label fields with this qualification. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The controlled vocabulary (obfuscation classes + sub-labels + field mapping + qualification rules) now lives in a `tag_vocabulary` MongoDB collection (single document, _id="current"), so it can be updated when the dataset changes without editing code. - app/services/tags.py: loads the active vocabulary from the DB (cached), falling back to the built-in DEFAULT_* baseline when the collection is empty (fresh DB / tests / DB down). Exposes accessor functions (get_tag_groups, get_classes, normalize_tags, dataset_sublabel_tag, get_sublabel_fields, get_dataset_url, reload_vocabulary) instead of module constants. - script/sync_tag_vocabulary.py: rebuilds the vocabulary document from the dataset (classes/sub-labels ordered by frequency, shared cipher names auto-detected and qualified) or seeds the built-in default. - Controllers, reviewer routes, and import_tags.py use the accessors; the tag vocabulary cache is reset between tests. - Tests for the default fallback and DB override. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#17) Replace the two-list transfer widget (tag change request modal + reviewer edit page) and the search page's multi-select dropdown with the same grouped checkbox layout used on the upload form: each class is a bold checkbox with its sub-labels indented underneath. - New shared partial templates/partial/tags_checkboxes.html (parametrized by input name + currently-checked tags) and static/js/tag_checkboxes.js (sub->parent auto-check), reused by upload, request modal, reviewer view/edit, and search. - Retire static/js/tag_transfer.js and its CSS. - Search uses checkboxes (in a scroll box); still GET/bookmarkable, still matches crackmes carrying all checked tags. - Tests updated for the checkbox markup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The modal-body scrolled as a whole (Spectre caps it at 50vh), so the optional reason box and Submit button could scroll out of view. Now only the checkbox list scrolls (.tags-scroll, max 50vh) while the reason field and submit stay pinned below it, with breathing room beneath the button. The dialog is also larger (900px / 94vw). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the shared tag checkbox helper so that, in addition to selecting a category when a technique is ticked and clearing techniques when a category is un-ticked, un-ticking the last remaining technique of a category now also un-ticks the category. Applies to the upload, request, and reviewer view/edit forms. Search intentionally keeps independent checkboxes (no auto-linking). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Browsers persist user-modified checkbox state across reloads, so reopening the "Request a tag change" dialog could show stale ticks on top of the crackme's actual tags. The dialog now resets to the crackme's current tags every time it opens (no draft kept between opens) and the form is marked autocomplete="off". Also add Undo (step back through changes) and Reset (back to the current tags) buttons to the dialog. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Like the tag request dialog, the reviewer Edit crackme form could show browser-restored (stale) field/checkbox state on reload or back-forward. Reset the form to the crackme's saved values on pageshow (load + bfcache) and mark it autocomplete="off". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace sync_tag_vocabulary.py + import_tags.py with a single script that runs both phases in the required order (vocabulary, then crackme tags), removing the footgun where re-tagging with a stale vocabulary silently dropped newly added tags. Flags: --vocab-only, --tags-only, --seed-default. The tags phase is now authoritative (clears then sets from the dataset) so crackmes dropped from the dataset also lose their tags. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements crackme obfuscation tags (closes #17) and makes tag/search results bookmarkable (closes #162).
What this adds
/search?tags=UPX). Tag chips link straight to a search.Vocabulary is data, not code
The controlled vocabulary lives in a
tag_vocabularyMongoDB collection so it can be updated without code changes when the dataset changes:script/sync_tag_vocabulary.pyrebuilds the vocabulary document from the dataset.script/import_tags.py(re)applies crackme tags.Notable UX details
Tests
227 passing, including new coverage for the vocabulary (default fallback + DB override), submission, GET-based tag search, tag change requests, and reviewer overrides.
🤖 Generated with Claude Code