mirror of
https://github.com/lintsinghua/DeepAudit.git
synced 2026-11-22 21:27:42 +08:00
338 lines
6.6 KiB
Markdown
338 lines
6.6 KiB
Markdown
# Testing Guide
|
|||
|
|
|
||
|
|
This document describes the testing strategy and how to run tests for XCodeReviewer.
|
||
|
|
|
||
|
|
## Test Structure
|
||
|
|
|
||
|
|
```
|
||
|
|
tests/
|
||
|
|
├── test_api_endpoints.py # API endpoint tests
|
||
|
|
├── test_e2e_workflows.py # End-to-end workflow tests
|
||
|
|
├── test_llm_service.py # LLM service tests
|
||
|
|
├── test_repository_scanner.py # Repository scanning tests
|
||
|
|
├── test_report_generation.py # Report generation tests
|
||
|
|
└── conftest.py # Shared fixtures
|
||
|
|
```
|
||
|
|
|
||
|
|
## Test Categories
|
||
|
|
|
||
|
|
### Unit Tests
|
||
|
|
Test individual components in isolation.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest tests/ -m unit
|
||
|
|
```
|
||
|
|
|
||
|
|
### Integration Tests
|
||
|
|
Test interactions between components.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest tests/ -m integration
|
||
|
|
```
|
||
|
|
|
||
|
|
### End-to-End Tests
|
||
|
|
Test complete user workflows.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest tests/ -m e2e
|
||
|
|
```
|
||
|
|
|
||
|
|
## Running Tests
|
||
|
|
|
||
|
|
### Run All Tests
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Using pytest directly
|
||
|
|
pytest
|
||
|
|
|
||
|
|
# Using test script
|
||
|
|
./scripts/run_tests.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run Specific Test File
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest tests/test_api_endpoints.py
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run Specific Test Class
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest tests/test_api_endpoints.py::TestAuthEndpoints
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run Specific Test
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest tests/test_api_endpoints.py::TestAuthEndpoints::test_register_user
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run with Coverage
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest --cov=. --cov-report=html
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run in Parallel
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest -n auto
|
||
|
|
```
|
||
|
|
|
||
|
|
## Coverage Reports
|
||
|
|
|
||
|
|
### Generate Coverage Report
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest --cov=. --cov-report=html --cov-report=term
|
||
|
|
```
|
||
|
|
|
||
|
|
### View HTML Report
|
||
|
|
|
||
|
|
```bash
|
||
|
|
open htmlcov/index.html # macOS
|
||
|
|
xdg-open htmlcov/index.html # Linux
|
||
|
|
start htmlcov/index.html # Windows
|
||
|
|
```
|
||
|
|
|
||
|
|
### Coverage Threshold
|
||
|
|
|
||
|
|
The project requires minimum 80% code coverage. Tests will fail if coverage is below this threshold.
|
||
|
|
|
||
|
|
## Test Fixtures
|
||
|
|
|
||
|
|
Common fixtures are defined in `conftest.py`:
|
||
|
|
|
||
|
|
- `db_session`: Database session for tests
|
||
|
|
- `test_client`: HTTP client for API tests
|
||
|
|
- `authenticated_user`: Pre-authenticated test user
|
||
|
|
- `auth_token`: Authentication token
|
||
|
|
|
||
|
|
## Writing Tests
|
||
|
|
|
||
|
|
### Unit Test Example
|
||
|
|
|
||
|
|
```python
|
||
|
|
def test_password_validation():
|
||
|
|
"""Test password validation logic"""
|
||
|
|
from core.security import validate_password
|
||
|
|
|
||
|
|
# Valid password
|
||
|
|
assert validate_password("SecurePass123!")
|
||
|
|
|
||
|
|
# Invalid password
|
||
|
|
assert not validate_password("weak")
|
||
|
|
```
|
||
|
|
|
||
|
|
### Integration Test Example
|
||
|
|
|
||
|
|
```python
|
||
|
|
@pytest.mark.asyncio
|
||
|
|
async def test_create_project(test_client, auth_token):
|
||
|
|
"""Test project creation via API"""
|
||
|
|
response = await test_client.post(
|
||
|
|
"/api/v1/projects",
|
||
|
|
headers={"Authorization": f"Bearer {auth_token}"},
|
||
|
|
json={
|
||
|
|
"name": "Test Project",
|
||
|
|
"source_type": "github"
|
||
|
|
}
|
||
|
|
)
|
||
|
|
assert response.status_code == 201
|
||
|
|
```
|
||
|
|
|
||
|
|
### E2E Test Example
|
||
|
|
|
||
|
|
```python
|
||
|
|
@pytest.mark.e2e
|
||
|
|
@pytest.mark.asyncio
|
||
|
|
async def test_complete_workflow(test_client, auth_token):
|
||
|
|
"""Test complete analysis workflow"""
|
||
|
|
# Create project
|
||
|
|
project = await create_project(test_client, auth_token)
|
||
|
|
|
||
|
|
# Create task
|
||
|
|
task = await create_task(test_client, auth_token, project["id"])
|
||
|
|
|
||
|
|
# Generate report
|
||
|
|
report = await generate_report(test_client, auth_token, task["id"])
|
||
|
|
|
||
|
|
assert report["status"] == "completed"
|
||
|
|
```
|
||
|
|
|
||
|
|
## Mocking
|
||
|
|
|
||
|
|
### Mock External Services
|
||
|
|
|
||
|
|
```python
|
||
|
|
from unittest.mock import patch, MagicMock
|
||
|
|
|
||
|
|
@patch('services.llm.openai_adapter.OpenAI')
|
||
|
|
def test_llm_call(mock_openai):
|
||
|
|
"""Test LLM service with mocked API"""
|
||
|
|
mock_openai.return_value.chat.completions.create.return_value = MagicMock(
|
||
|
|
choices=[MagicMock(message=MagicMock(content="Test response"))]
|
||
|
|
)
|
||
|
|
|
||
|
|
# Test code here
|
||
|
|
```
|
||
|
|
|
||
|
|
### Mock Database
|
||
|
|
|
||
|
|
```python
|
||
|
|
@pytest.fixture
|
||
|
|
async def mock_db():
|
||
|
|
"""Mock database session"""
|
||
|
|
from unittest.mock import AsyncMock
|
||
|
|
|
||
|
|
session = AsyncMock()
|
||
|
|
session.commit = AsyncMock()
|
||
|
|
session.rollback = AsyncMock()
|
||
|
|
|
||
|
|
return session
|
||
|
|
```
|
||
|
|
|
||
|
|
## Continuous Integration
|
||
|
|
|
||
|
|
Tests run automatically on:
|
||
|
|
- Pull requests
|
||
|
|
- Commits to main branch
|
||
|
|
- Scheduled daily runs
|
||
|
|
|
||
|
|
### GitHub Actions Workflow
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
name: Tests
|
||
|
|
|
||
|
|
on: [push, pull_request]
|
||
|
|
|
||
|
|
jobs:
|
||
|
|
test:
|
||
|
|
runs-on: ubuntu-latest
|
||
|
|
steps:
|
||
|
|
- uses: actions/checkout@v2
|
||
|
|
- name: Set up Python
|
||
|
|
uses: actions/setup-python@v2
|
||
|
|
with:
|
||
|
|
python-version: '3.11'
|
||
|
|
- name: Install dependencies
|
||
|
|
run: pip install -r requirements.txt
|
||
|
|
- name: Run tests
|
||
|
|
run: ./scripts/run_tests.sh
|
||
|
|
- name: Upload coverage
|
||
|
|
uses: codecov/codecov-action@v2
|
||
|
|
```
|
||
|
|
|
||
|
|
## Performance Testing
|
||
|
|
|
||
|
|
### Load Testing with Locust
|
||
|
|
|
||
|
|
```python
|
||
|
|
from locust import HttpUser, task, between
|
||
|
|
|
||
|
|
class APIUser(HttpUser):
|
||
|
|
wait_time = between(1, 3)
|
||
|
|
|
||
|
|
@task
|
||
|
|
def list_projects(self):
|
||
|
|
self.client.get("/api/v1/projects")
|
||
|
|
```
|
||
|
|
|
||
|
|
Run load test:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
locust -f tests/load_test.py --host=http://localhost:8000
|
||
|
|
```
|
||
|
|
|
||
|
|
## Security Testing
|
||
|
|
|
||
|
|
### SQL Injection Tests
|
||
|
|
|
||
|
|
```python
|
||
|
|
def test_sql_injection_protection(test_client, auth_token):
|
||
|
|
"""Test SQL injection protection"""
|
||
|
|
response = test_client.get(
|
||
|
|
"/api/v1/projects",
|
||
|
|
params={"search": "'; DROP TABLE projects; --"}
|
||
|
|
)
|
||
|
|
assert response.status_code != 500
|
||
|
|
```
|
||
|
|
|
||
|
|
### Authentication Tests
|
||
|
|
|
||
|
|
```python
|
||
|
|
def test_unauthorized_access(test_client):
|
||
|
|
"""Test unauthorized access is blocked"""
|
||
|
|
response = test_client.get("/api/v1/projects")
|
||
|
|
assert response.status_code == 401
|
||
|
|
```
|
||
|
|
|
||
|
|
## Debugging Tests
|
||
|
|
|
||
|
|
### Run with Debug Output
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest -vv --tb=long
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run with PDB
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest --pdb
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run Specific Failed Tests
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pytest --lf # Last failed
|
||
|
|
pytest --ff # Failed first
|
||
|
|
```
|
||
|
|
|
||
|
|
## Best Practices
|
||
|
|
|
||
|
|
1. **Test Naming**: Use descriptive names that explain what is being tested
|
||
|
|
2. **Arrange-Act-Assert**: Structure tests clearly
|
||
|
|
3. **One Assertion**: Focus each test on one behavior
|
||
|
|
4. **Independent Tests**: Tests should not depend on each other
|
||
|
|
5. **Fast Tests**: Keep unit tests fast (< 1 second)
|
||
|
|
6. **Mock External Services**: Don't make real API calls in tests
|
||
|
|
7. **Clean Up**: Use fixtures to clean up test data
|
||
|
|
8. **Coverage**: Aim for 80%+ coverage
|
||
|
|
9. **Documentation**: Add docstrings to test functions
|
||
|
|
10. **Markers**: Use pytest markers to categorize tests
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
### Tests Hanging
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Add timeout
|
||
|
|
pytest --timeout=30
|
||
|
|
```
|
||
|
|
|
||
|
|
### Database Conflicts
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Use separate test database
|
||
|
|
export TEST_DATABASE_URL=sqlite+aiosqlite:///./test.db
|
||
|
|
```
|
||
|
|
|
||
|
|
### Import Errors
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Add project root to PYTHONPATH
|
||
|
|
export PYTHONPATH="${PYTHONPATH}:$(pwd)"
|
||
|
|
```
|
||
|
|
|
||
|
|
### Async Tests Failing
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Ensure pytest-asyncio is installed
|
||
|
|
pip install pytest-asyncio
|
||
|
|
```
|
||
|
|
|
||
|
|
## Resources
|
||
|
|
|
||
|
|
- [Pytest Documentation](https://docs.pytest.org/)
|
||
|
|
- [Coverage.py Documentation](https://coverage.readthedocs.io/)
|
||
|
|
- [Testing Best Practices](https://docs.python-guide.org/writing/tests/)
|