>_

Troubleshooting

This page consolidates common issues and solutions across the Open Privacy Suite stack.


Docker Issues

Containers not starting

# Check container status
docker-compose ps

# View logs for a specific service
docker-compose logs postgres
docker-compose logs proxy-backend
docker-compose logs anvil

# Restart a specific service
docker-compose restart proxy-backend

# Full reset (destroys volumes)
docker-compose down -v
docker-compose up -d

Container health check failing

If proxy-backend stays in "starting" state, check that its dependencies are healthy:

docker-compose logs postgres | tail -20
docker-compose logs anvil | tail -20

The backend waits for both PostgreSQL and the Ethereum node to be reachable before starting.

Docker network conflicts

If port bindings fail, check for other services using the same ports:

lsof -i :8080  # Open Privacy Suite
lsof -i :5173  # Frontend
lsof -i :5432  # PostgreSQL
lsof -i :8545  # Anvil / Ethereum node

block-explorer-frontend in restart loop: host not found in upstream "block-explorer-api"

Privacy-mode full-stack only. The block-explorer-frontend container is configured to attach to both public and bff-zone networks (docker-compose.privacy.dev.yml), but on rare daemon-state glitches docker connects only one of the two at startup. With bff-zone missing, nginx inside the container can't resolve block-explorer-api and the container crashes-and-restarts forever.

Confirm:

docker inspect privacy-proxy-block-explorer-frontend-1 \
  --format '{{range $net,$_ := .NetworkSettings.Networks}}{{$net}} {{end}}'
# Expected:  privacy-proxy_bff-zone privacy-proxy_public
# Broken:    privacy-proxy_public                          ← only one

Fix — force-recreate just the frontend (no data loss, the explorer postgres stays up):

set -a && source .env.privacy.dev && set +a
docker compose -f docker-compose.privacy.dev.yml \
  up -d --force-recreate --no-deps block-explorer-frontend

If the issue recurs on every make full-stack-dev, file a ticket — durable causes can include compose-version differences or a stale network entry left behind by a previous compose project. The compose config on main is correct; the symptom is daemon-side state, not config.


Database Issues

Cannot connect to PostgreSQL

# Check if PostgreSQL container is running
docker-compose ps postgres

# Connect manually
docker-compose exec postgres psql -U postgres -d privacy_proxy

# Check connection string
echo $DATABASE_URL

Migration failures

# View migration status
docker-compose logs proxy-backend | grep -i migrat

# Reset database completely (destroys all data)
docker-compose down -v
docker-compose up -d

Data loss

Running docker-compose down -v destroys all database volumes. All organizations, users, policies, and audit logs will be lost.

testcontainers failures in unit tests

If unit tests fail to start PostgreSQL via testcontainers:

  1. Verify Docker is running: docker info
  2. Check Docker socket permissions
  3. Set TEST_DATABASE_URL to use an external PostgreSQL instead:
export TEST_DATABASE_URL="postgres://postgres:postgres@localhost:5432/privacy_proxy_test?sslmode=disable"
go test ./internal/...

Authentication Issues

ErrorStatusCauseSolution
missing Authorization header401No Bearer token in requestAdd Authorization: Bearer <token> header
invalid token401Malformed or corrupted JWTRe-authenticate to get a new token
session not found or expired401Auth session expired (10-min TTL)Start a new authentication flow
JWZ verification failed401Invalid ZK proofEnsure wallet is using correct credentials and network
humanity_verification_required403User lacks ProofOfHumanity credentialComplete PoH verification at Billions
account is banned403Banned user on login / refresh (on the /rpc data path a ban is masked as an opaque 404 method not found)Contact org admin to remove ban
(opaque method not found)404On the /rpc path, a non-KYC'd user's call is masked as method not found — the proxy does not disclose the reasonComplete KYC verification

Token expiration

Access tokens expire after 5 minutes (production; dev/mockauth builds use 30 minutes). Refresh tokens expire after 7 days.

# Refresh an expired access token
curl -X POST http://localhost:8080/refresh \
  -H "Content-Type: application/json" \
  -d '{"refresh_token": "YOUR_REFRESH_TOKEN"}'

If the refresh token is also expired, re-authenticate through the web UI.

Azure AD authentication failures

ErrorCauseSolution
No Microsoft tab on loginEnv vars missingCheck AZURE_AD_CLIENT_ID and AZURE_AD_CLIENT_SECRET in .env
403 "tenant is not authorized"Tenant not in allowlistAdd tenant via POST /api/v1/admin/azure-tenants with X-Admin-Token
AADSTS50011 "redirect URI mismatch"Wrong URI in Azure PortalSet to http://localhost:5173/auth/azure/callback
"nonce mismatch"State expiredRetry -- state tokens have 10-minute TTL

RBAC Issues

ErrorStatusCauseSolution
method not found404The JSON-RPC method is not in the user's allowlist, or the target contract is not owned by the user's org — direct RBAC denials (method and contract) are masked as an opaque 404, never a 403Check group permissions and contract grants in the admin UI.
transaction denied: cross-org access not permitted403A send transaction's runtime trace enters a contract owned by another orgOnly interact with contracts owned by your organization.
call denied: cross-org access not permitted403An eth_call resolved at runtime to an internal call into a foreign-org contract (e.g. a same-org wrapper STATICCALLing another org's contract).Inspect the trace; either the target contract reaches into another org or a proxy upgrade has re-targeted it. Update grants or stop calling that path.
call denied: trace depth exceeded; not provable as same-org403The eth_call trace exceeded the maximum recursion depth, so deeper call targets could not be validated. Failed closed.Reduce nesting in the called contract or split into multiple calls.
call denied: tracing temporarily unavailable403Upstream debug_traceCall failed (timeout, connection refused, malformed response). Failed closed.Check upstream node health and debug namespace exposure. Re-try; if persistent, check the proxy logs for the underlying error.
call denied: invalid request shape400The to address is malformed, or the user-supplied from does not match any of the caller's linked Ethereum addresses (potential spoof attempt).Validate the request body. The proxy rebinds from only to JWT-linked EOAs.
deploy claim required403User lacks deploy permissionAsk org admin to add deploy claim to user's group.
admin claim required403Operation requires admin privilegesAsk org admin to grant admin claim.
upgrade claim required403Proxy upgrade requires upgrade permissionAsk org admin to grant upgrade claim.

Claims hierarchy

Remember that the admin claim has implied permissions:

  • admin implies deploy and upgrade

The legacy read / write claims were removed — non-privileged method access is now gated solely by the per-group method allowlist (GroupAccess.allowed_methods). Don't try to add read or write as claims; the resolver no longer recognizes them.

Blocked methods

The following method families are always blocked regardless of permissions:

debug_*    admin_*    personal_*    miner_*
txpool_*   clique_*   les_*         eth_sign
eth_signTransaction   eth_subscribe  eth_unsubscribe

Multicall blocked

Calls to known Multicall contract addresses are blocked because they allow batching arbitrary contract calls, bypassing method-level restrictions:

  • Multicall3: 0xcA11bde05977b3631167028862bE2a173976CA11
  • Multicall2: 0x5ba1e12693dc8f9c48aad8770482f4739beed696
  • Multicall1: 0xeefba1e63905ef1d7acba5a8513c70307c1ce441

Runtime tracing as defense-in-depth

When runtime tracing is enabled, all internal calls are validated regardless of the target address, on both the write side (eth_sendTransaction/eth_sendRawTransaction) and the read side (eth_call). Even custom multicall contracts (not in the hardcoded list) are caught because every CALL, DELEGATECALL, and STATICCALL target is verified.


Port Conflicts

PortServiceResolution
8080Open Privacy SuiteStop other HTTP servers or change PORT env var
5173Frontend (Vite)Stop other Vite instances or change Vite config port
5432PostgreSQLStop local PostgreSQL: brew services stop postgresql (macOS) or sudo systemctl stop postgresql (Linux)
8545Anvil / Ethereum nodeStop other Anvil or Hardhat instances

Build Issues

Go build failures

# Clean module cache
go clean -modcache

# Re-download dependencies
go mod download

# Verify dependencies
go mod verify

Frontend build failures

cd frontend

# Clear node_modules and reinstall
rm -rf node_modules
npm install

# Clear Vite cache
rm -rf node_modules/.vite
npm run dev

Docker build failures

# Rebuild without cache
docker-compose build --no-cache

# Pull latest base images
docker-compose pull

Rate Limiting

If you are receiving 429 Too Many Requests:

  • Determine whether the response is a local concurrency rejection or an upstream rate-limit/circuit-breaker rejection by checking proxy metrics and logs.
  • For a local concurrency rejection, reduce simultaneous in-flight requests and retry after existing requests finish.
  • For an upstream rejection, check the rate tier associated with the group's rpc_api_key (or the global fallback key) in the upstream RPC proxy.
  • Respect Retry-After when present. Changing group permissions inside Open Privacy Suite does not change the upstream API key's quota.

Getting Help

If your issue is not covered here:

  1. Check the backend logs: docker-compose logs -f proxy-backend
  2. Run tests with verbose output: go test -v ./e2e/...
  3. Check the GitHub Issues for known problems.