Skip to content

Authentication is usually the first dependency every other service discovers during an incident.

An application can degrade when a reporting job is down. A background worker can retry when a queue is slow. But if the identity system is unavailable, users cannot start sessions, administrators cannot reach operational tools, and downstream services may lose the one issuer they are designed to trust.

That is why backup and recovery for an authentication system should not be treated as a database administration footnote. It is part of the production operating model. The question is not only “do we have backups?” It is whether the team can restore the identity boundary, prove it works, and communicate clearly while every dependent application is waiting.

Protect the state that defines trust

In ProAuth, the backup scope starts with the main ProAuth database. The backup and recovery documentation describes it as the place where core configuration lives: identity provider configuration, client registrations, API scopes and resources, system settings, permissions, and roles.

That makes it more than an application database. It contains the configuration that decides which customers can authenticate, which applications may request tokens, which resources are known, and which administrators can operate the platform.

The second major scope is every User Store database. User Store databases contain user-related information such as profiles, credentials, claims, and consent records. ProAuth supports multiple User Store configurations: a User Store can have its own database, and multiple User Store instances can also share one database. The recovery plan has to enumerate the actual topology, not the idealized one from the original deployment diagram.

Then come the keys. ProAuth protects sensitive database values with external encryption keys through Data Protection and X.509 certificates. A restored database without the matching encryption keys is not a recovered system. The backup plan must include the certificates, certificate passwords, and key-rotation history needed to decrypt data written before the incident.

Finally, include the operational surface that shapes the login experience: custom views, custom assets, labels, configuration exports, and deployment settings. ProAuth custom assets are managed at global, subscription, client application, and tenant levels and served through the /assets/ endpoint. Custom authentication views and assets may look like branding, but during recovery they are part of the customer-facing authentication surface.

Database consistency matters more than backup volume

Most teams can produce a list of backup jobs. Fewer can explain whether the restored ProAuth database and the restored User Store databases represent the same point in time.

That distinction matters because identity state crosses database boundaries. The ProAuth backup guidance recommends transaction-consistent backups across the main ProAuth database and User Store databases when possible. If backups are taken at different times, ProAuth provides mechanisms to re-synchronize User Store data with the main database through API calls after recovery.

That gives operations teams a practical rule: design for consistency first, and document the reconciliation path when consistency cannot be guaranteed.

flowchart TD
    incident["Identity incident or data-loss scenario"]
    isolate["Freeze or isolate affected writes"]
    restoreMain["Restore ProAuth main database"]
    restoreStores["Restore every User Store database"]
    keys["Recover encryption certificates and passwords"]
    start["Start ProAuth in recovery environment"]
    verifyDb["Verify application database connectivity"]
    consistent{"Backups are transaction-consistent?"}
    resync["Run User Store synchronization API"]
    validate["Validate IdP health, login, token, admin, and asset flows"]
    communicate["Publish recovery status and follow-up actions"]

    incident --> isolate
    isolate --> restoreMain
    isolate --> restoreStores
    isolate --> keys
    restoreMain --> start
    restoreStores --> start
    keys --> start
    start --> verifyDb
    verifyDb --> consistent
    consistent -->|"Yes"| validate
    consistent -->|"No"| resync
    resync --> validate
    validate --> communicate

The restore sequence should be rehearsed outside production. A rehearsal should prove that backups are usable, that ProAuth can access restored databases, that encryption keys can decrypt protected data, and that the system behaves correctly after recovery.

Do not confuse rebuild automation with recovery

Configuration exports are useful. They help reviewers understand the intended state of tenants, identity providers, client applications, roles, custom views, custom assets, and labels. They also give delivery teams a way to compare environments and rebuild controlled configuration after a mistake.

They are not a replacement for database backups.

The difference is scope. A configuration export can describe what should exist. A database backup also preserves operational state: user records, consent records, credentials in User Stores, audit tables, and other data that cannot be recreated from a clean configuration file. ProAuth’s Management API and automation tooling are important for controlled changes, but the recovery objective still depends on restorable databases and restorable encryption keys.

Use exports as supporting evidence:

Recovery assetWhy it mattersRehearsal check
ProAuth main database backupRestores identity provider configuration, client registrations, scopes, resources, system settings, permissions, and rolesRestore to an isolated environment and confirm ProAuth can read configuration
User Store database backupsRestores user profiles, credentials, claims, and consent recordsRestore every configured User Store and verify login for a test tenant
Encryption certificates and passwordsAllows ProAuth to read encrypted values in restored databasesStart ProAuth with restored keys and validate protected data access
Configuration exports or API snapshotsDocuments intended tenant, IdP, client, role, view, asset, and label configurationCompare restored configuration with the reviewed source of truth
Custom assets and viewsPreserves the customer-facing authentication experienceLoad branded login, MFA, consent, and password recovery pages
Audit trail dataSupports incident review and change investigationConfirm audit entries are accessible only to authorized roles
Monitoring and logsShows whether backup jobs, restore steps, and recovered services are healthyConfirm alerts, telemetry, and dashboards reflect the recovery environment

The checklist is intentionally blunt. Recovery succeeds when the team can prove the restored identity service is usable, not when the backup dashboard shows green.

Treat auxiliary services according to their blast radius

ProAuth uses auxiliary services for pub/sub messaging, caching, and session state storage. The backup and recovery documentation is clear about their relative criticality: losing that state is not critical to system integrity. Users may need to sign in again, tokens may be treated as revoked, session state may be lost, and temporary caches can be rebuilt.

That does not mean auxiliary services are irrelevant. It means they belong in a different recovery category from databases and encryption keys. Back them up when it reduces user disruption, but do not let cache recovery delay the restoration of the identity configuration, User Store data, and encryption material that ProAuth needs to operate correctly.

This is where recovery objectives become concrete. A short RTO for the login path may require rebuilding caches and sessions from scratch. A stricter user-experience objective may justify more work to preserve session-related state. The right answer depends on the business requirement, but the distinction should be written down before an incident.

Validate the recovered identity boundary

After the system starts, recovery is not finished.

Start with infrastructure checks. ProAuth exposes health checks for startup, liveness, and readiness in the application. For identity provider configuration, the IdP instance health endpoint reports whether an instance has configuration errors and whether it is operational. For a User Store IdP, operational status includes database connectivity and schema checks. For an OpenID Connect IdP, it includes whether the authority URL is valid and whether OpenID Connect configuration can be retrieved.

Then validate user-facing and application-facing behavior:

  • Can a user sign in through a representative User Store tenant?
  • Can a user sign in through a representative federated IdP?
  • Can an application complete the expected OpenID Connect flow and receive tokens from ProAuth?
  • Can an administrator access the management surface with the expected role?
  • Do custom login views, custom assets, and recovery pages render correctly?
  • Are audit trail entries available to SystemAdmin or AuditTrailReader roles only?
  • Are logs, metrics, and traces flowing to the monitoring system?

The monitoring documentation describes ProAuth observability through logs, metrics, and traces, with exporters such as OTLP, Prometheus, console, and structured JSON logs. Use that telemetry during rehearsal, not only during real incidents. A restore that works silently is harder to trust than a restore that produces the expected operational signals.

Plan the communication path before the incident

Authentication incidents create pressure because many teams feel the outage at once. Application owners see failed sign-ins. Support teams receive user reports. Customer administrators may not know whether the problem is their IdP, ProAuth, or the SaaS application.

The recovery plan should therefore include communication triggers, not just technical steps:

  • who declares an identity incident
  • who owns the ProAuth recovery runbook
  • when dependent application teams are notified
  • which customer-facing message is used for login disruption
  • when administrators should stop making configuration changes
  • how the team records recovered backup timestamps and known data gaps
  • who approves return to normal operation

Those details are easy to dismiss during calm weeks. During a recovery, they decide whether the organization is coordinated or merely busy.

Rehearse recovery as an operating capability

A backup that has never been restored is an assumption. A recovery plan that has never been rehearsed is a draft.

For ProAuth, the rehearsal should cover the main database, every User Store database, encryption certificates, configuration exports, custom assets, custom views, audit evidence, health checks, and monitoring. It should also include the awkward parts: inconsistent backup timestamps, expired certificates, missing passwords, stale configuration exports, and unclear ownership between identity, platform, database, and customer support teams.

The goal is not to make identity incidents pleasant. It is to make them bounded. When authentication is the shared production dependency, recovery has to be practiced like an application release: documented, repeatable, observable, and owned.

For the product-specific backup sequence, start with the ProAuth Backup and Recovery documentation. For supporting operations, review Monitoring, Auditing, Auth View Customization, and Custom Assets.