Self-Hosting Guide

Overview

Many organizations publish statistical indicators derived from survey data (unemployment rates, poverty rates, innovation metrics) without disclosing how those indicators were computed. The final number is public, but the methodology – which variables were used, what transformations were applied, what survey design was used – remains opaque.

metasurvey solves this by separating private data from public methodology. You can deploy the metasurvey API on your own infrastructure where:

Microdata stays private: raw survey files never leave your network.
Indicators are public: computed results (point estimates, standard errors, confidence intervals) are served via the API.
Methodology is transparent: anyone can query the recipe (data transformation steps) and the workflow (estimation calls and survey design) that produced each indicator.

The traceability chain is:

Indicator                Workflow                    Recipe
(the number)      -->    (how it was estimated)  --> (how variables were built)
value: 0.082             svymean(~pd, design)        step_compute(svy, pd = ...)
se: 0.003                estimation_type: annual     step_recode(svy, pea, ...)
cv: 0.037                recipe_ids: [ech_emp_001]   depends_on: [e27, f66, ...]

Architecture

The system has two R services: a public API that serves indicators and their traceability, and a private Worker that has metasurvey installed and access to microdata. The worker is the only component that touches raw survey data.

metasurvey self-hosting infrastructure

Key separation: The Worker loads microdata, fetches recipes from MongoDB, applies them (bake_steps), runs the estimation (workflow), and posts the result back to the API. The public API never touches microdata – it only serves pre-computed indicators and their traceability.

The frontend can also request on-demand computations with filters (e.g., “unemployment rate for women in Montevideo”) via POST /indicators/compute, which the API proxies to the Worker.

Quick Start with Docker Compose

The repository includes a docker-compose.yml that starts MongoDB, the plumber API, and the Shiny recipe explorer. No external database required.

1. Clone and configure

git clone https://github.com/metasurveyr/metasurvey.git
cd metasurvey

Create a .env file (or use the defaults for development):

# .env
MONGO_USER=metasurvey
MONGO_PASSWORD=change-me-in-production
METASURVEY_JWT_SECRET=change-me-in-production
METASURVEY_ADMIN_EMAIL=admin@example.com

2. Start the stack

docker compose up --build

This starts four services:

Service	URL	Description
`mongo`	`localhost:27017`	MongoDB 7 with persistent volume
`worker`	`localhost:8788`	Compute worker (internal only)
`api`	`http://localhost:8787`	Plumber REST API
`shiny`	`http://localhost:3838`	Recipe explorer

3. Initialize the database

# Create collections and indexes
docker compose exec mongo mongosh \
  -u metasurvey -p change-me-in-production \
  --authenticationDatabase admin \
  metasurvey /dev/stdin < inst/scripts/setup_mongodb.js

# Seed example recipes, workflows, and indicators
docker compose exec api Rscript -e '
  Sys.setenv(
    METASURVEY_MONGO_URI = "mongodb://metasurvey:change-me-in-production@mongo:27017/?authSource=admin"
  )
  source("/app/seed_ech_recipes.R")
'

4. Verify

curl http://localhost:8787/health

{
  "status": "ok",
  "service": "metasurvey-api",
  "mongodb": "connected"
}

Computing and Publishing Indicators

The typical flow: load survey data, apply a recipe, run a workflow, and publish the result as an indicator with full traceability.

library(metasurvey)

# 1. Load survey data (private -- stays on your server)
svy <- Survey$new(
  data = my_survey_data,
  edition = "2024",
  type = "ech",
  engine = "data.table",
  weight = add_weight(annual = "W_ANO")
)

# 2. Apply a recipe (defines variables like unemployment status)
svy <- step_compute(svy,
  pd = data.table::fcase(
    pobpcoac == 2, 1L,
    pobpcoac %in% c(1, 3), 0L
  ),
  comment = "Unemployed: POBPCOAC == 2"
)
svy <- bake_steps(svy)

# 3. Run the estimation (workflow)
result <- workflow(
  svy = list(svy),
  survey::svymean(~pd, na.rm = TRUE),
  estimation_type = "annual"
)

# result is a data.table:
#       stat      value     se     cv
# svymean: pd    0.082  0.003  0.037

Now publish the indicator to the API:

# Connect to your local API
configure_api("http://localhost:8787")
api_login("admin@example.com", "your-password")

# Build the indicator payload
indicator <- list(
  name = "Unemployment Rate 2024",
  description = "Annual unemployment rate, population 14+",
  recipe_id = "ech_employment_001",
  workflow_id = "ech_wf_labor",
  survey_type = "ech",
  edition = "2024",
  estimation_type = "annual",
  stat = result$stat[1],
  value = result$value[1],
  se = result$se[1],
  cv = result$cv[1],
  confint_lower = result$confint_lower[1],
  confint_upper = result$confint_upper[1],
  metadata = list(
    formula = "~pd",
    estimation_function = "svymean"
  )
)

# Publish (requires authentication)
resp <- httr2::request("http://localhost:8787/indicators") |>
  httr2::req_headers(
    Authorization = paste("Bearer", Sys.getenv("METASURVEY_TOKEN"))
  ) |>
  httr2::req_body_json(indicator) |>
  httr2::req_perform()

httr2::resp_body_json(resp)
# {ok: true, id: "ind_1708099200_42"}

Consuming Indicators (Transparency)

Once published, indicators and their full methodology are accessible without authentication. This is the transparency layer.

Get the indicator value

curl http://localhost:8787/indicators/ind_ech_unemployment_2024

{
  "ok": true,
  "indicator": {
    "id": "ind_ech_unemployment_2024",
    "name": "Tasa de desempleo 2024",
    "value": 0.082,
    "se": 0.003,
    "cv": 0.037,
    "confint_lower": 0.076,
    "confint_upper": 0.088,
    "survey_type": "ech",
    "edition": "2024",
    "recipe_id": "ech_employment_001",
    "workflow_id": "ech_wf_labor"
  }
}

Get the workflow (how it was estimated)

curl http://localhost:8787/indicators/ind_ech_unemployment_2024/workflow

{
  "ok": true,
  "indicator_id": "ind_ech_unemployment_2024",
  "workflow": {
    "id": "ech_wf_labor",
    "name": "Mercado Laboral ECH",
    "estimation_type": "annual",
    "recipe_ids": ["ech_pobpcoac_000", "ech_employment_001"],
    "calls": [
      "svymean(~pea, design, na.rm=TRUE)",
      "svymean(~po, design, na.rm=TRUE)",
      "svymean(~pd, design, na.rm=TRUE)"
    ],
    "call_metadata": [
      {
        "type": "svymean",
        "formula": "~pea",
        "description": "Tasa de actividad"
      },
      {
        "type": "svymean",
        "formula": "~pd",
        "description": "Tasa de desempleo"
      }
    ]
  }
}

The workflow tells you what statistical function was used (svymean), what formula (~pd), and which recipes were applied to the data before estimation.

Get the recipe (how variables were built)

curl http://localhost:8787/indicators/ind_ech_unemployment_2024/recipe

{
  "ok": true,
  "indicator_id": "ind_ech_unemployment_2024",
  "recipe": {
    "id": "ech_employment_001",
    "name": "Employment Status",
    "steps": [
      "step_compute(svy, po = fcase(pobpcoac == 1, 1L, TRUE, 0L), comment = 'Employed')",
      "step_compute(svy, pd = fcase(pobpcoac == 2, 1L, TRUE, 0L), comment = 'Unemployed')",
      "step_compute(svy, pea = fcase(pobpcoac %in% 1:2, 1L, TRUE, 0L), comment = 'EAP')"
    ],
    "depends_on": ["pobpcoac"],
    "doc": {
      "input_variables": ["pobpcoac"],
      "output_variables": ["po", "pd", "pea"],
      "pipeline": [
        {"step": 1, "type": "compute", "outputs": ["po"]},
        {"step": 2, "type": "compute", "outputs": ["pd"]},
        {"step": 3, "type": "compute", "outputs": ["pea"]}
      ]
    }
  }
}

The recipe tells you exactly how each variable was constructed from the original survey variables. Combined with the workflow, anyone can verify the full computation chain without accessing the microdata.

Indicator API Reference

Method	Endpoint	Auth	Description
`GET`	`/indicators`	No	List and search published indicators
`GET`	`/indicators/:id`	No	Get a single indicator with metadata
`GET`	`/indicators/:id/recipe`	No	Get the recipe that built the variables
`GET`	`/indicators/:id/workflow`	No	Get the workflow (estimation + design)
`POST`	`/indicators`	Yes	Publish a computed indicator
`POST`	`/indicators/compute`	Yes	On-demand estimation via the Worker (with filters)

Query parameters for `GET /indicators`

Parameter	Type	Description
`search`	string	Regex search on indicator name
`survey_type`	string	Filter by survey type
`recipe_id`	string	Filter by recipe ID
`workflow_id`	string	Filter by workflow ID
`edition`	string	Filter by survey edition
`limit`	integer	Max results (default 50)
`offset`	integer	Skip N results (default 0)

Indicator document fields

Field	Type	Required	Description
`id`	string	Auto	Unique identifier
`name`	string	Yes	Indicator name
`workflow_id`	string	Yes	Workflow that produced it
`value`	number	Yes	Point estimate
`recipe_id`	string	No	Recipe that built variables
`description`	string	No	Human-readable description
`survey_type`	string	No	Survey type
`edition`	string	No	Survey edition
`estimation_type`	string	No	`annual`, `quarterly`, `monthly`
`stat`	string	No	Statistic label (e.g., `svymean: pd`)
`se`	number	No	Standard error
`cv`	number	No	Coefficient of variation
`confint_lower`	number	No	Lower bound of confidence interval
`confint_upper`	number	No	Upper bound of confidence interval
`metadata`	object	No	Additional context (formula, notes)
`published_at`	string	Auto	ISO timestamp

Deploying with Minikube

For Kubernetes-based deployment, you can test locally with Minikube.

Start Minikube

minikube start

MongoDB deployment

# k8s/mongo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: metasurvey-mongo
  namespace: metasurvey
spec:
  replicas: 1
  selector:
    matchLabels:
      app: metasurvey-mongo
  template:
    metadata:
      labels:
        app: metasurvey-mongo
    spec:
      containers:
        - name: mongo
          image: mongo:7
          ports:
            - containerPort: 27017
          env:
            - name: MONGO_INITDB_ROOT_USERNAME
              value: metasurvey
            - name: MONGO_INITDB_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: mongo-password
---
apiVersion: v1
kind: Service
metadata:
  name: mongo-service
  namespace: metasurvey
spec:
  selector:
    app: metasurvey-mongo
  ports:
    - port: 27017
      targetPort: 27017

Worker deployment

# k8s/worker.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: metasurvey-worker
  namespace: metasurvey
spec:
  replicas: 1
  selector:
    matchLabels:
      app: metasurvey-worker
  template:
    metadata:
      labels:
        app: metasurvey-worker
    spec:
      containers:
        - name: worker
          image: ghcr.io/metasurveyr/metasurvey-worker:latest
          ports:
            - containerPort: 8788
          env:
            - name: METASURVEY_MONGO_URI
              value: "mongodb://metasurvey:$(MONGO_PASSWORD)@mongo-service:27017/?authSource=admin"
            - name: MONGO_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: mongo-password
          volumeMounts:
            - name: survey-data
              mountPath: /data/surveys
              readOnly: true
          resources:
            requests:
              memory: "1Gi"
              cpu: "500m"
            limits:
              memory: "4Gi"
              cpu: "2000m"
      volumes:
        - name: survey-data
          persistentVolumeClaim:
            claimName: survey-microdata-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: worker-service
  namespace: metasurvey
spec:
  selector:
    app: metasurvey-worker
  ports:
    - port: 8788
      targetPort: 8788

API deployment

# k8s/api.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: metasurvey-api
  namespace: metasurvey
spec:
  replicas: 2
  selector:
    matchLabels:
      app: metasurvey-api
  template:
    metadata:
      labels:
        app: metasurvey-api
    spec:
      containers:
        - name: api
          image: ghcr.io/metasurveyr/metasurvey-api:latest
          ports:
            - containerPort: 8787
          env:
            - name: METASURVEY_MONGO_URI
              value: "mongodb://metasurvey:$(MONGO_PASSWORD)@mongo-service:27017/?authSource=admin"
            - name: MONGO_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: mongo-password
            - name: METASURVEY_JWT_SECRET
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: jwt-secret
            - name: METASURVEY_WORKER_URL
              value: "http://worker-service:8788"
          livenessProbe:
            httpGet:
              path: /health
              port: 8787
            initialDelaySeconds: 30
          readinessProbe:
            httpGet:
              path: /health
              port: 8787
            initialDelaySeconds: 10
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1Gi"
              cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: metasurvey-api
  namespace: metasurvey
spec:
  type: NodePort
  selector:
    app: metasurvey-api
  ports:
    - port: 8787
      targetPort: 8787

Apply and test

kubectl create namespace metasurvey

kubectl create secret generic metasurvey-secrets \
  --namespace metasurvey \
  --from-literal=mongo-password=change-me \
  --from-literal=jwt-secret=change-me

kubectl apply -f k8s/mongo.yaml -f k8s/worker.yaml -f k8s/api.yaml

# Access the API
minikube service metasurvey-api -n metasurvey

Deploying on AWS (EKS)

For production on AWS, use EKS with DocumentDB (MongoDB compatible) as the managed database.

1. Create the EKS cluster

# Requires: aws-cli, eksctl
eksctl create cluster \
  --name metasurvey-prod \
  --region sa-east-1 \
  --node-type t3.medium \
  --nodes 2 \
  --nodes-min 1 \
  --nodes-max 5

2. Create DocumentDB (MongoDB compatible)

aws docdb create-db-cluster \
  --db-cluster-identifier metasurvey-db \
  --engine docdb \
  --master-username metasurvey_admin \
  --master-user-password "$DB_PASSWORD" \
  --vpc-security-group-ids "$SG_ID" \
  --db-subnet-group-name "$SUBNET_GROUP"

aws docdb create-db-instance \
  --db-instance-identifier metasurvey-db-1 \
  --db-cluster-identifier metasurvey-db \
  --db-instance-class db.r6g.large \
  --engine docdb

3. Configure secrets in Kubernetes

aws eks update-kubeconfig --name metasurvey-prod --region sa-east-1

kubectl create namespace metasurvey

kubectl create secret generic metasurvey-secrets \
  --namespace metasurvey \
  --from-literal=mongo-uri="mongodb://metasurvey_admin:$DB_PASSWORD@metasurvey-db.cluster-xxx.sa-east-1.docdb.amazonaws.com:27017/?tls=true&tlsCAFile=/rds-combined-ca-bundle.pem&retryWrites=false" \
  --from-literal=jwt-secret="$(openssl rand -hex 32)"

4. Deployment with ALB Ingress

# k8s/aws-api.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: metasurvey-api
  namespace: metasurvey
spec:
  replicas: 2
  selector:
    matchLabels:
      app: metasurvey-api
  template:
    metadata:
      labels:
        app: metasurvey-api
    spec:
      containers:
        - name: api
          image: ghcr.io/metasurveyr/metasurvey-api:latest
          ports:
            - containerPort: 8787
          env:
            - name: METASURVEY_MONGO_URI
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: mongo-uri
            - name: METASURVEY_JWT_SECRET
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: jwt-secret
            - name: METASURVEY_WORKER_URL
              value: "http://worker-service:8788"
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1Gi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 8787
            initialDelaySeconds: 30
          readinessProbe:
            httpGet:
              path: /health
              port: 8787
            initialDelaySeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: metasurvey-api
  namespace: metasurvey
spec:
  type: ClusterIP
  selector:
    app: metasurvey-api
  ports:
    - port: 8787
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: metasurvey-ingress
  namespace: metasurvey
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/certificate-arn: "$ACM_CERT_ARN"
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
spec:
  rules:
    - host: api.metasurvey.example.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: metasurvey-api
                port:
                  number: 8787

# Install AWS Load Balancer Controller (if not installed)
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=metasurvey-prod

kubectl apply -f k8s/aws-api.yaml

Deploying on Azure (AKS)

For production on Azure, use AKS with CosmosDB (MongoDB API) as the managed database.

1. Create the AKS cluster

# Requires: az cli
az group create --name rg-metasurvey --location brazilsouth

az aks create \
  --resource-group rg-metasurvey \
  --name aks-metasurvey \
  --node-count 2 \
  --node-vm-size Standard_B2ms \
  --generate-ssh-keys \
  --enable-managed-identity

2. Create CosmosDB with MongoDB API

az cosmosdb create \
  --name cosmos-metasurvey \
  --resource-group rg-metasurvey \
  --kind MongoDB \
  --capabilities EnableMongo \
  --default-consistency-level Session \
  --locations regionName=brazilsouth

# Get connection string
COSMOS_URI=$(az cosmosdb keys list \
  --name cosmos-metasurvey \
  --resource-group rg-metasurvey \
  --type connection-strings \
  --query "connectionStrings[0].connectionString" -o tsv)

3. Configure secrets in Kubernetes

az aks get-credentials \
  --resource-group rg-metasurvey \
  --name aks-metasurvey

kubectl create namespace metasurvey

kubectl create secret generic metasurvey-secrets \
  --namespace metasurvey \
  --from-literal=mongo-uri="$COSMOS_URI" \
  --from-literal=jwt-secret="$(openssl rand -hex 32)"

4. Deployment with NGINX Ingress

# k8s/azure-api.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: metasurvey-api
  namespace: metasurvey
spec:
  replicas: 2
  selector:
    matchLabels:
      app: metasurvey-api
  template:
    metadata:
      labels:
        app: metasurvey-api
    spec:
      containers:
        - name: api
          image: ghcr.io/metasurveyr/metasurvey-api:latest
          ports:
            - containerPort: 8787
          env:
            - name: METASURVEY_MONGO_URI
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: mongo-uri
            - name: METASURVEY_JWT_SECRET
              valueFrom:
                secretKeyRef:
                  name: metasurvey-secrets
                  key: jwt-secret
            - name: METASURVEY_WORKER_URL
              value: "http://worker-service:8788"
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1Gi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 8787
            initialDelaySeconds: 30
          readinessProbe:
            httpGet:
              path: /health
              port: 8787
            initialDelaySeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: metasurvey-api
  namespace: metasurvey
spec:
  type: ClusterIP
  selector:
    app: metasurvey-api
  ports:
    - port: 8787
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: metasurvey-ingress
  namespace: metasurvey
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - api.metasurvey.example.org
      secretName: metasurvey-tls
  rules:
    - host: api.metasurvey.example.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: metasurvey-api
                port:
                  number: 8787

# Install NGINX Ingress Controller
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace

# Install cert-manager for TLS
helm repo add jetstack https://charts.jetstack.io
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

kubectl apply -f k8s/azure-api.yaml

Feature Flags

The API supports feature flags via environment variables to control which modules are available. This lets you run the same API image in different configurations:

Variable	Default	Description
`METASURVEY_ENABLE_INDICATORS`	`1`	Enable `/indicators` endpoints
`METASURVEY_ENABLE_WORKER`	`0`	Enable `POST /indicators/compute` (on-demand)

For example, the public metasurvey API on Railway only serves recipes and workflows – indicators and worker are disabled:

METASURVEY_ENABLE_INDICATORS=0
METASURVEY_ENABLE_WORKER=0

A self-hosted deployment with full capabilities:

METASURVEY_ENABLE_INDICATORS=1
METASURVEY_ENABLE_WORKER=1
METASURVEY_WORKER_URL=http://worker:8788

Hybrid Deployment: Public Registry + Private Infrastructure

You don’t need to run your own MongoDB to access shared recipes. You can deploy your own API + Worker pointing to the public metasurvey recipe registry, so that researchers with access to microdata can apply community-published recipes locally.

The organization deploys two services:

plumber API (:8787) – connects to the public MongoDB to read recipes and workflows. Receives requests from the frontend or researchers, and proxies compute requests to the worker.
Worker (:8788) – has metasurvey installed and access to private microdata. Fetches recipes from the public registry (via MongoDB), applies them, and runs estimations.

  Public metasurvey MongoDB             Your infrastructure
  (recipes, workflows)                  (private network)
  +-------------------+           +---------------------------+
  |  MongoDB Atlas     |<----------| plumber API  :8787        |
  |  (read-only)       |           |   - reads recipes/wf      |
  +-------------------+           |   - proxies POST /compute  |
                                   +-------------+-------------+
                                                 |
                                    POST /compute|
                                                 v
                                   +-------------+-------------+
                                   | Worker  :8788              |
                                   |   - loads microdata        |
                                   |   - applies recipes        |
                                   |   - runs workflow()        |
                                   +---------------------------+
                                                 ^
                                   +-------------+-------------+
                                   | Microdata (.sav, .dta)    |
                                   | PRIVATE — never leaves    |
                                   +---------------------------+

# .env for hybrid deployment
# Both API and Worker point to the public metasurvey MongoDB
METASURVEY_MONGO_URI=mongodb+srv://readonly:public@metasurvey.mongodb.net
METASURVEY_DB=metasurvey

# Worker has local microdata
SURVEY_DATA_PATH=/secure/microdata

# API proxies compute requests to the worker
METASURVEY_WORKER_URL=http://worker:8788
METASURVEY_ENABLE_WORKER=1

# Indicators disabled (no local MongoDB to store them)
METASURVEY_ENABLE_INDICATORS=0

docker compose up api worker

In this mode:

Recipes and workflows are fetched from the public registry – any recipe published by the community is available to your API and worker.
Microdata stays private – the worker loads survey files from a local volume, never exposed externally.
Estimation is local – the API receives the request, proxies it to the worker, which applies recipes and runs workflow() on your infrastructure.
No MongoDB to maintain – you use the public registry as-is.

This is useful for research teams that have access to restricted microdata and want to apply standardized recipes without maintaining their own database.

Production Deployment

For production deployments with Terraform modules (AWS/Azure/GCP), Helm charts, and CI/CD pipelines, see the infrastructure repository:

The infrastructure repository will include:

Terraform modules for managed databases (RDS, DocumentDB/CosmosDB, Cloud SQL, Memorystore)
Kubernetes manifests with autoscaling, TLS ingress, and secrets management
Docker image builds and registry configuration
Monitoring and health check setup

Next Steps

API and Database Reference – Full endpoint documentation, MongoDB schema, and authentication details
Creating and Sharing Recipes – Build recipes for reproducible survey processing
Estimation Workflows – Compute weighted survey estimates with workflow()

Overview

Architecture

Quick Start with Docker Compose

1. Clone and configure

2. Start the stack

3. Initialize the database

4. Verify

Computing and Publishing Indicators

Consuming Indicators (Transparency)

Get the indicator value

Get the workflow (how it was estimated)

Get the recipe (how variables were built)

Indicator API Reference

Query parameters for GET /indicators

Indicator document fields

Deploying with Minikube

Start Minikube

MongoDB deployment

Worker deployment

API deployment

Apply and test

Deploying on AWS (EKS)

1. Create the EKS cluster

2. Create DocumentDB (MongoDB compatible)

3. Configure secrets in Kubernetes

4. Deployment with ALB Ingress

Deploying on Azure (AKS)

1. Create the AKS cluster

2. Create CosmosDB with MongoDB API

3. Configure secrets in Kubernetes

4. Deployment with NGINX Ingress

Feature Flags

Hybrid Deployment: Public Registry + Private Infrastructure

Production Deployment

Next Steps

Query parameters for `GET /indicators`