Troubleshooting
This page consolidates common issues and solutions across the Open Privacy Suite stack.
Docker Issues
Containers not starting
# Check container status
docker-compose ps
# View logs for a specific service
docker-compose logs postgres
docker-compose logs proxy-backend
docker-compose logs anvil
# Restart a specific service
docker-compose restart proxy-backend
# Full reset (destroys volumes)
docker-compose down -v
docker-compose up -d
Container health check failing
If proxy-backend stays in "starting" state, check that its dependencies are healthy:
docker-compose logs postgres | tail -20
docker-compose logs anvil | tail -20
The backend waits for both PostgreSQL and the Ethereum node to be reachable before starting.
Docker network conflicts
If port bindings fail, check for other services using the same ports:
lsof -i :8080 # Open Privacy Suite
lsof -i :5173 # Frontend
lsof -i :5432 # PostgreSQL
lsof -i :8545 # Anvil / Ethereum node
block-explorer-frontend in restart loop: host not found in upstream "block-explorer-api"
Privacy-mode full-stack only. The block-explorer-frontend container is configured to attach to both public and bff-zone networks (docker-compose.privacy.dev.yml), but on rare daemon-state glitches docker connects only one of the two at startup. With bff-zone missing, nginx inside the container can't resolve block-explorer-api and the container crashes-and-restarts forever.
Confirm:
docker inspect privacy-proxy-block-explorer-frontend-1 \
--format '{{range $net,$_ := .NetworkSettings.Networks}}{{$net}} {{end}}'
# Expected: privacy-proxy_bff-zone privacy-proxy_public
# Broken: privacy-proxy_public ← only one
Fix — force-recreate just the frontend (no data loss, the explorer postgres stays up):
set -a && source .env.privacy.dev && set +a
docker compose -f docker-compose.privacy.dev.yml \
up -d --force-recreate --no-deps block-explorer-frontend
If the issue recurs on every make full-stack-dev, file a ticket — durable causes can include compose-version differences or a stale network entry left behind by a previous compose project. The compose config on main is correct; the symptom is daemon-side state, not config.
Database Issues
Cannot connect to PostgreSQL
# Check if PostgreSQL container is running
docker-compose ps postgres
# Connect manually
docker-compose exec postgres psql -U postgres -d privacy_proxy
# Check connection string
echo $DATABASE_URL
Migration failures
# View migration status
docker-compose logs proxy-backend | grep -i migrat
# Reset database completely (destroys all data)
docker-compose down -v
docker-compose up -d
Data loss
Running docker-compose down -v destroys all database volumes. All organizations, users, policies, and audit logs will be lost.
testcontainers failures in unit tests
If unit tests fail to start PostgreSQL via testcontainers:
- Verify Docker is running:
docker info - Check Docker socket permissions
- Set
TEST_DATABASE_URLto use an external PostgreSQL instead:
export TEST_DATABASE_URL="postgres://postgres:postgres@localhost:5432/privacy_proxy_test?sslmode=disable"
go test ./internal/...
Authentication Issues
| Error | Status | Cause | Solution |
|---|---|---|---|
missing Authorization header | 401 | No Bearer token in request | Add Authorization: Bearer <token> header |
invalid token | 401 | Malformed or corrupted JWT | Re-authenticate to get a new token |
session not found or expired | 401 | Auth session expired (10-min TTL) | Start a new authentication flow |
JWZ verification failed | 401 | Invalid ZK proof | Ensure wallet is using correct credentials and network |
humanity_verification_required | 403 | User lacks ProofOfHumanity credential | Complete PoH verification at Billions |
account is banned | 403 | Banned user on login / refresh (on the /rpc data path a ban is masked as an opaque 404 method not found) | Contact org admin to remove ban |
(opaque method not found) | 404 | On the /rpc path, a non-KYC'd user's call is masked as method not found — the proxy does not disclose the reason | Complete KYC verification |
Token expiration
Access tokens expire after 5 minutes (production; dev/mockauth builds use 30 minutes). Refresh tokens expire after 7 days.
# Refresh an expired access token
curl -X POST http://localhost:8080/refresh \
-H "Content-Type: application/json" \
-d '{"refresh_token": "YOUR_REFRESH_TOKEN"}'
If the refresh token is also expired, re-authenticate through the web UI.
Azure AD authentication failures
| Error | Cause | Solution |
|---|---|---|
| No Microsoft tab on login | Env vars missing | Check AZURE_AD_CLIENT_ID and AZURE_AD_CLIENT_SECRET in .env |
| 403 "tenant is not authorized" | Tenant not in allowlist | Add tenant via POST /api/v1/admin/azure-tenants with X-Admin-Token |
| AADSTS50011 "redirect URI mismatch" | Wrong URI in Azure Portal | Set to http://localhost:5173/auth/azure/callback |
| "nonce mismatch" | State expired | Retry -- state tokens have 10-minute TTL |
RBAC Issues
| Error | Status | Cause | Solution |
|---|---|---|---|
method not found | 404 | The JSON-RPC method is not in the user's allowlist, or the target contract is not owned by the user's org — direct RBAC denials (method and contract) are masked as an opaque 404, never a 403 | Check group permissions and contract grants in the admin UI. |
transaction denied: cross-org access not permitted | 403 | A send transaction's runtime trace enters a contract owned by another org | Only interact with contracts owned by your organization. |
call denied: cross-org access not permitted | 403 | An eth_call resolved at runtime to an internal call into a foreign-org contract (e.g. a same-org wrapper STATICCALLing another org's contract). | Inspect the trace; either the target contract reaches into another org or a proxy upgrade has re-targeted it. Update grants or stop calling that path. |
call denied: trace depth exceeded; not provable as same-org | 403 | The eth_call trace exceeded the maximum recursion depth, so deeper call targets could not be validated. Failed closed. | Reduce nesting in the called contract or split into multiple calls. |
call denied: tracing temporarily unavailable | 403 | Upstream debug_traceCall failed (timeout, connection refused, malformed response). Failed closed. | Check upstream node health and debug namespace exposure. Re-try; if persistent, check the proxy logs for the underlying error. |
call denied: invalid request shape | 400 | The to address is malformed, or the user-supplied from does not match any of the caller's linked Ethereum addresses (potential spoof attempt). | Validate the request body. The proxy rebinds from only to JWT-linked EOAs. |
deploy claim required | 403 | User lacks deploy permission | Ask org admin to add deploy claim to user's group. |
admin claim required | 403 | Operation requires admin privileges | Ask org admin to grant admin claim. |
upgrade claim required | 403 | Proxy upgrade requires upgrade permission | Ask org admin to grant upgrade claim. |
Claims hierarchy
Remember that the admin claim has implied permissions:
adminimpliesdeployandupgrade
The legacy read / write claims were removed — non-privileged method access is now gated solely by the per-group method allowlist (GroupAccess.allowed_methods). Don't try to add read or write as claims; the resolver no longer recognizes them.
Blocked methods
The following method families are always blocked regardless of permissions:
debug_* admin_* personal_* miner_*
txpool_* clique_* les_* eth_sign
eth_signTransaction eth_subscribe eth_unsubscribe
Multicall blocked
Calls to known Multicall contract addresses are blocked because they allow batching arbitrary contract calls, bypassing method-level restrictions:
- Multicall3:
0xcA11bde05977b3631167028862bE2a173976CA11 - Multicall2:
0x5ba1e12693dc8f9c48aad8770482f4739beed696 - Multicall1:
0xeefba1e63905ef1d7acba5a8513c70307c1ce441
Runtime tracing as defense-in-depth
When runtime tracing is enabled, all internal calls are validated regardless of the target address, on both the write side (eth_sendTransaction/eth_sendRawTransaction) and the read side (eth_call). Even custom multicall contracts (not in the hardcoded list) are caught because every CALL, DELEGATECALL, and STATICCALL target is verified.
Port Conflicts
| Port | Service | Resolution |
|---|---|---|
| 8080 | Open Privacy Suite | Stop other HTTP servers or change PORT env var |
| 5173 | Frontend (Vite) | Stop other Vite instances or change Vite config port |
| 5432 | PostgreSQL | Stop local PostgreSQL: brew services stop postgresql (macOS) or sudo systemctl stop postgresql (Linux) |
| 8545 | Anvil / Ethereum node | Stop other Anvil or Hardhat instances |
Build Issues
Go build failures
# Clean module cache
go clean -modcache
# Re-download dependencies
go mod download
# Verify dependencies
go mod verify
Frontend build failures
cd frontend
# Clear node_modules and reinstall
rm -rf node_modules
npm install
# Clear Vite cache
rm -rf node_modules/.vite
npm run dev
Docker build failures
# Rebuild without cache
docker-compose build --no-cache
# Pull latest base images
docker-compose pull
Rate Limiting
If you are receiving 429 Too Many Requests:
- Determine whether the response is a local concurrency rejection or an upstream rate-limit/circuit-breaker rejection by checking proxy metrics and logs.
- For a local concurrency rejection, reduce simultaneous in-flight requests and retry after existing requests finish.
- For an upstream rejection, check the rate tier associated with the group's
rpc_api_key(or the global fallback key) in the upstream RPC proxy. - Respect
Retry-Afterwhen present. Changing group permissions inside Open Privacy Suite does not change the upstream API key's quota.
Getting Help
If your issue is not covered here:
- Check the backend logs:
docker-compose logs -f proxy-backend - Run tests with verbose output:
go test -v ./e2e/... - Check the GitHub Issues for known problems.