Please check out the contribution guide (CONTRIBUTION.md) before making any contributions.

PendingChangesBot

PendingChangesBot is a Django application that inspects pending changes on Wikimedia projects using the Flagged Revisions API. It fetches the 50 oldest pending pages for a selected wiki, caches their pending revisions together with editor metadata, and exposes a Vue.js interface for reviewing the results.

✨ Now ready for Toolforge deployment! See TOOLFORGE_DEPLOYMENT.md for deployment instructions.

Installation

1. Fork the repository

   • A fork is a new repository that shares code and visibility settings with the original “upstream” repository. Forks are often used to iterate on ideas or changes before they are proposed back to the upstream repository.
   • For more details about how to fork a repository, please check out the github docs for it.

2. Clone the repository

   • Using SSH (requires setup of ssh keys)

   git clone git@github.com:Wikimedia-Suomi/PendingChangesBot-ng.git
   cd PendingChangesBot-ng

   • Using HTTPS

   git clone https://github.com/Wikimedia-Suomi/PendingChangesBot-ng.git
   cd PendingChangesBot-ng

3. Check your python version (recommended)

   • On Windows:

   python --version

   • On macOS / Linux:

   python3 --version

   Install if not found *for python3 you need to install pip3

4. Create and activate a virtual environment (recommended)

   python3 -m venv venv
   source venv/bin/activate  # On Windows use: .venv\\Scripts\\activate

5. Install Python dependencies

   • On Windows:

   pip install --upgrade pip
   pip install -r requirements.txt

   • On macOS / Linux:

   pip3 install --upgrade pip
   pip3 install -r requirements.txt

6. Install pre-commit hooks (recommended for contributors)

   pre-commit install

   This will automatically format and lint your code before each commit.

Quick Start for Sublime Text Users

If you prefer using Sublime Text instead of VS Code:

• Open the repository folder in Sublime Text.
• Ensure your virtual environment is activated in the terminal inside Sublime Text.
• Use the terminal or Sublime's build system to run Django commands, for example:

python manage.py runserver

You can install Sublime Text packages for Python linting and formatting to complement pre-commit hooks.

Troubleshooting Tips Windows venv activation: If .venv\Scripts\activate doesn't work, try running PowerShell as Administrator or use:

source .venv/Scripts/activate

pip errors: If installing dependencies fails, ensure your pip is upgraded:

python -m pip install --upgrade pip

Port conflicts: If runserver complains that port 8000 is in use, run:

python manage.py runserver 8080

Configuring Pywikibot

Pywikibot is used to interact with MediaWiki APIs (FlaggedRevs, page info, etc.). Statistics data is fetched via direct SQL connections to wiki replica databases (Superset is no longer used).

1. Move to app directory All pywikibot and manage.py commands should be run in the app directory.

   cd app

2. Create a Pywikibot configuration

   echo "usernames['meta']['meta'] = 'WIKIMEDIA_USERNAME'" > user-config.py

3. Log in with Pywikibot

   • Using management command

   python manage.py auth_with_username_and_password

   • On Windows:

   python -m pywikibot.scripts.login -site:meta

   • On macOS / Linux:

   python3 -m pywikibot.scripts.login -site:meta

   The command should report Logged in on metawiki and create a persistent login cookie at ~/.pywikibot/pywikibot.lwp.

Running the database migrations

cd app

On Windows:

python manage.py makemigrations
python manage.py migrate

• On macOS / Linux:

python3 manage.py makemigrations
python3 manage.py migrate

Running the application

The Django project serves both the API and the Vue.js frontend from the same codebase.

cd app

• On Windows:

python manage.py runserver

• On macOS / Linux:

python3 manage.py runserver

Open http://127.0.0.1:8000/ in your browser to use the interface. JSON endpoints are available under /api/wikis/<wiki_id>/…, for example /api/wikis/1/pending/.

API Documentation

Interactive API documentation is available via Swagger UI:

• Swagger UI: http://127.0.0.1:8000/swagger/ - Interactive API explorer
• ReDoc: http://127.0.0.1:8000/redoc/ - Alternative documentation view

For detailed API documentation, see docs/API_DOCUMENTATION.md.

Statistics

The application includes statistics functionality that tracks FlaggedRevs data and reviewer activity using direct SQL access to wiki replica databases.

Features:

• Monthly aggregates (total pages, reviewed pages, pending lag)
• Reviewer activity metrics
• Individual review records with delay calculations
• Interactive charts and visualizations

Loading Statistics:

# Load FlaggedRevs statistics for Finnish Wikipedia
TOOLFORGE_DEPLOYMENT=true python manage.py load_flaggedrevs_statistics_direct_sql --wiki fi

# Load individual review records
TOOLFORGE_DEPLOYMENT=true python manage.py load_review_statistics_direct_sql --wiki fi --limit 10000

For detailed documentation on the statistics implementation, see docs/DIRECT_SQL_STATISTICS.md.

Running unit tests

Unit tests live in the Django backend project. Run them from the app/ directory so Django can locate the correct settings module.

cd app

• On Windows:

python manage.py test

• On macOS / Linux:

python3 manage.py test

Code Coverage

Run tests with coverage measurement:

cd app
coverage run --source='.' manage.py test

View coverage report in terminal:

coverage report

Generate and view HTML coverage report:

coverage html
open htmlcov/index.html  # On macOS
# Or navigate to htmlcov/index.html in your browser

Code Quality & Security

This project uses automated checks to catch bugs and security issues before they reach production.

Tools

• mypy - Type checking to catch type errors before runtime
• Ruff (Bandit rules) - Security scanning for common vulnerabilities
• pip-audit - Dependency vulnerability scanning

All checks run automatically in CI on every PR. You can also run them locally for faster feedback.

Running Checks Locally (Optional)

Option 1: Run all checks at once

./scripts/run-checks.sh

Option 2: Run individually

# Type checking
cd app && python -m mypy reviews --config-file=../pyproject.toml

# Security scanning
python -m ruff check --select S app/

# Dependency scanning
python -m pip_audit -r requirements.txt

The CI will run these same checks on every PR.

Code Formatting and Linting

This project uses Ruff for code formatting and linting.

Note: If you installed pre-commit hooks (step 6 above), formatting and linting happen automatically before each commit. You don't need to run these commands manually.

Manual Commands

# Format code
ruff format app/

# Check and fix linting issues
ruff check app/ --fix

If you are working inside a virtual environment, ensure it is activated before executing the command.