Files

338 lines
6.6 KiB
Markdown
Raw Permalink Normal View History

2025-11-01 23:46:29 +08:00
# Testing Guide
This document describes the testing strategy and how to run tests for XCodeReviewer.
## Test Structure
```
tests/
├── test_api_endpoints.py # API endpoint tests
├── test_e2e_workflows.py # End-to-end workflow tests
├── test_llm_service.py # LLM service tests
├── test_repository_scanner.py # Repository scanning tests
├── test_report_generation.py # Report generation tests
└── conftest.py # Shared fixtures
```
## Test Categories
### Unit Tests
Test individual components in isolation.
```bash
pytest tests/ -m unit
```
### Integration Tests
Test interactions between components.
```bash
pytest tests/ -m integration
```
### End-to-End Tests
Test complete user workflows.
```bash
pytest tests/ -m e2e
```
## Running Tests
### Run All Tests
```bash
# Using pytest directly
pytest
# Using test script
./scripts/run_tests.sh
```
### Run Specific Test File
```bash
pytest tests/test_api_endpoints.py
```
### Run Specific Test Class
```bash
pytest tests/test_api_endpoints.py::TestAuthEndpoints
```
### Run Specific Test
```bash
pytest tests/test_api_endpoints.py::TestAuthEndpoints::test_register_user
```
### Run with Coverage
```bash
pytest --cov=. --cov-report=html
```
### Run in Parallel
```bash
pytest -n auto
```
## Coverage Reports
### Generate Coverage Report
```bash
pytest --cov=. --cov-report=html --cov-report=term
```
### View HTML Report
```bash
open htmlcov/index.html # macOS
xdg-open htmlcov/index.html # Linux
start htmlcov/index.html # Windows
```
### Coverage Threshold
The project requires minimum 80% code coverage. Tests will fail if coverage is below this threshold.
## Test Fixtures
Common fixtures are defined in `conftest.py`:
- `db_session`: Database session for tests
- `test_client`: HTTP client for API tests
- `authenticated_user`: Pre-authenticated test user
- `auth_token`: Authentication token
## Writing Tests
### Unit Test Example
```python
def test_password_validation():
"""Test password validation logic"""
from core.security import validate_password
# Valid password
assert validate_password("SecurePass123!")
# Invalid password
assert not validate_password("weak")
```
### Integration Test Example
```python
@pytest.mark.asyncio
async def test_create_project(test_client, auth_token):
"""Test project creation via API"""
response = await test_client.post(
"/api/v1/projects",
headers={"Authorization": f"Bearer {auth_token}"},
json={
"name": "Test Project",
"source_type": "github"
}
)
assert response.status_code == 201
```
### E2E Test Example
```python
@pytest.mark.e2e
@pytest.mark.asyncio
async def test_complete_workflow(test_client, auth_token):
"""Test complete analysis workflow"""
# Create project
project = await create_project(test_client, auth_token)
# Create task
task = await create_task(test_client, auth_token, project["id"])
# Generate report
report = await generate_report(test_client, auth_token, task["id"])
assert report["status"] == "completed"
```
## Mocking
### Mock External Services
```python
from unittest.mock import patch, MagicMock
@patch('services.llm.openai_adapter.OpenAI')
def test_llm_call(mock_openai):
"""Test LLM service with mocked API"""
mock_openai.return_value.chat.completions.create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="Test response"))]
)
# Test code here
```
### Mock Database
```python
@pytest.fixture
async def mock_db():
"""Mock database session"""
from unittest.mock import AsyncMock
session = AsyncMock()
session.commit = AsyncMock()
session.rollback = AsyncMock()
return session
```
## Continuous Integration
Tests run automatically on:
- Pull requests
- Commits to main branch
- Scheduled daily runs
### GitHub Actions Workflow
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.11'
- name: Install dependencies
run: pip install -r requirements.txt
- name: Run tests
run: ./scripts/run_tests.sh
- name: Upload coverage
uses: codecov/codecov-action@v2
```
## Performance Testing
### Load Testing with Locust
```python
from locust import HttpUser, task, between
class APIUser(HttpUser):
wait_time = between(1, 3)
@task
def list_projects(self):
self.client.get("/api/v1/projects")
```
Run load test:
```bash
locust -f tests/load_test.py --host=http://localhost:8000
```
## Security Testing
### SQL Injection Tests
```python
def test_sql_injection_protection(test_client, auth_token):
"""Test SQL injection protection"""
response = test_client.get(
"/api/v1/projects",
params={"search": "'; DROP TABLE projects; --"}
)
assert response.status_code != 500
```
### Authentication Tests
```python
def test_unauthorized_access(test_client):
"""Test unauthorized access is blocked"""
response = test_client.get("/api/v1/projects")
assert response.status_code == 401
```
## Debugging Tests
### Run with Debug Output
```bash
pytest -vv --tb=long
```
### Run with PDB
```bash
pytest --pdb
```
### Run Specific Failed Tests
```bash
pytest --lf # Last failed
pytest --ff # Failed first
```
## Best Practices
1. **Test Naming**: Use descriptive names that explain what is being tested
2. **Arrange-Act-Assert**: Structure tests clearly
3. **One Assertion**: Focus each test on one behavior
4. **Independent Tests**: Tests should not depend on each other
5. **Fast Tests**: Keep unit tests fast (< 1 second)
6. **Mock External Services**: Don't make real API calls in tests
7. **Clean Up**: Use fixtures to clean up test data
8. **Coverage**: Aim for 80%+ coverage
9. **Documentation**: Add docstrings to test functions
10. **Markers**: Use pytest markers to categorize tests
## Troubleshooting
### Tests Hanging
```bash
# Add timeout
pytest --timeout=30
```
### Database Conflicts
```bash
# Use separate test database
export TEST_DATABASE_URL=sqlite+aiosqlite:///./test.db
```
### Import Errors
```bash
# Add project root to PYTHONPATH
export PYTHONPATH="${PYTHONPATH}:$(pwd)"
```
### Async Tests Failing
```bash
# Ensure pytest-asyncio is installed
pip install pytest-asyncio
```
## Resources
- [Pytest Documentation](https://docs.pytest.org/)
- [Coverage.py Documentation](https://coverage.readthedocs.io/)
- [Testing Best Practices](https://docs.python-guide.org/writing/tests/)