Version: 9.2

Deployment on AWS

This page describes the process and requirements for deploying a new Qrvey MultiPlatform Environment in AWS.

Note: For v9 installations, contact Customer Support to obtain your Docker registry credentials.

Requirements

Docker: The latest version of Docker should be installed.
The Docker Image for the desired version, found in the release notes.
The registry username and password provided by the Qrvey Support team.
IAM user with Admin access, an access key, and a secret key: This is needed to create the resources for deployment.
The VPC (or equivalent) that is being used to deploy the Qrvey Platform should have a minimum CIDR of /22.
An S3 Bucket to store the state file. It should be in the same region as the deployment.
SMTP configuration to send emails.
A DNS Hosted Zone (Optional): To generate valid SSL Certificates for the Qrvey Composer domain. If there is no domain setup, Qrvey generates one with the following format: $deployment_id.mp.qrveyapp.com. To automatically set up a custom DNS, the Route 53 zone should be in the same account as the deployment, and credentials should have sufficient permissions.

If using an IAM user for deployment, here are the minimum required permissions for deployment:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*",
                "elasticloadbalancing:*",
                "autoscaling:*",
                "eks:*",
                "iam:*",
                "route53:*",
                "s3:*",
                "secretsmanager:*",
                "rds:*",
                "rds-db:*",
                "kms:*",
                "cloudwatch:*",
                "logs:*",
                "acm:*",
                "elasticfilesystem:*",
                "ecr:*",
                "ecr-public:*",
                "events:*",
                "ssm:*",
                "sts:*",
                "sqs:*",
                "dynamodb:*",
                "vpce:*",       
                "opensearch:*",
                "cloudfront:CreateCloudFrontOriginAccessIdentity",
                "athena:StartQueryExecution",
                "athena:GetQueryExecution",
                "athena:GetQueryResults",
                "athena:GetDatabase",
                "athena:CreateDataCatalog",
                "glue:CreateDatabase",
                "glue:GetDatabase",
                "glue:GetDatabases",
                "geo:*",
                "geo-places:*",
                "geo-routes:*"
            ],
            "Resource": "*"
        }
    ]
}

Installation

To install Qrvey 9.x.x in your AWS account, create the following file: config.json.

For more information, see Configuration Variables.

{
    "account_config": {
        "access_key_id": "<ACCESS_KEY>",
        "secret_access_key": "<SECRET_KEY>>",
        "region": "<REGION>",
        "bucket": "<S3_BUCKET_TO_STORE_THE_STATE_FILE>",
        "key": "<FILE_NAME>"
    },
    "variables": {
        "registry_user": "<REGISTRY_USER_PROVIDED_BY_QRVEY_SUPPORT>",
        "registry_key": "<REGISTRY_KEY_PROVIDED_BY_QRVEY_SUPPORT>",
        "qrvey_chart_version": "<QRVEY_VERSION>", // found at the end of the docker image provided above under pre-requisites
        "enable_location_services": true,
        "es_config": {
            "size": "large", // can be small, medium, or large
            "count": 1
        },
        "customer_info": {
            "firstname": "",
            "lastname": "",
            "email": "email@company.com",
            "company": "<COMPANY_NAME>"
        },
        "initial_admin_email": "admin@company.tld",
        "globalization": {
            "google_client_email": "",          // optional
            "google_client_private_key": "",    // optional
            "google_document_id": "",           // optional
            "google_document_sheet_title": ""   // optional
        },
        "enable_trino": false                   // optional
    }
}

When the above prerequisites are ready, run the following commands to install Qrvey.

From your terminal, navigate to the directory that contains the config file above.

Use the following command to log in to the Qrvey Registry.

docker login qrvey.azurecr.io --username $registry_user --password-stdin <<< $registry_key

Run the installation commands with the desired Terraform option: plan, apply, output, or destroy.
For installation, use the apply option. The installation process should take about two hours.
```
# This command is for MAC. Choose the platform param as required based on your OS.
docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} apply
```
After running the apply command, wait until the process is complete and review the resources created.

You can run the following command to get environment outputs, including the admin username and password, to log in to Qrvey. The following command is for MAC. Set the --platform param as required based on your OS.*

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} output

    ##########
   ###      ####
 ###          ###    +++ +++      +++  +++++  +++      ++
###            ### ++++   +++    +++ +++   +++ ++     +++
###            ### ++      ++   +++ +++   ++++  ++   +++
###            ### ++       ++  ++  ++++++++    +++  ++
 ###          ###  ++       ++++++  +++          ++++++
   ####    #####   ++        ++++    +++   +++    ++++
     ########      ++         ++      +++++++     +++
       #####                                      ++
        ########                               ++++

# ENVIRONMENT DETAILS
DEPLOYMENT_ID: deployment-id
URL: https://deployment-id.mp.qrveyapp.com
ADMIN URL: https://deployment-id.mp.qrveyapp.com/admin/app/
ADMIN USER: admin@company.tld
ADMIN PASSWORD: generated_admin_password
APIKEY: qrvey_api_key
PostgresqlConnection: postgres://qrvey_usr:db_password@deployment-id-qrvey-db.postgres.database.azure.com:5432/postgres
ES_HOST: https://1.2.3.4:9200/
ES_USERNAME: elastic
ES_PASSWORD: elastic_password

Navigate to your Qrvey domain and log in to the platform.

Note: To use a custom domain for your deployment, set the property "dns_zone_name" under the "variables" object in your config.json to the desired URL. After deployment, you receive a Load Balancer URL in the output. Set this Load Balancer URL as the target for the CNAME record of your custom domain in your DNS provider.

Upgrade to a Newer Version

An upgrade uses similar steps to an installation.

Before You Begin

Before upgrading from a version earlier than 9.2.1 to 9.2.1 or later, add the --refresh-helm flag to the apply command. This cleans up removed microservices, but can cause a few minutes of downtime. Plan to perform upgrades during off hours.
Before upgrading from version 9.1.x to 9.2.2 or later, it is recommended to set RabbitMQ replicas to 1. Use the flag --force-rabbitmq with --refresh-helm to reset the RabbitMQ replicas. To keep the replicas at 3, use the rabbitmq_replica_count: 3 in the config.JSON file.
Before upgrading from v9.0.x or v9.1.x to v9.2 or later and switching to use a single-AZ configuration after upgrading, review Switch from Multi-AZ to Single-AZ Configuration.
Before upgrading from v9.0.x to v9.2.2, verify whether data-load performance needs to be improved. This can be done using the dataload_config property.

Perform the Upgrade

To upgrade your Qrvey MultiPlatform Environment to a newer version, follow the same steps as in the Installation section above. The only change required is in Step 1:

Update the qrvey_chart_version variable in your config.json file to the new version.

After updating the version, repeat Steps 2–6 from the Installation section, using the updated qrvey_version value in the relevant commands.

This applies the upgrade and update your environment to the specified version.

Switch from Multi-AZ to Single-AZ Configuration

If you are upgrading from v9.0.x or v9.1.x to v9.2 or later, note that previous versions used a multi-AZ configuration, where nodes could be distributed across different Availability Zones (AZs). This can result in intra-AZ data transfer costs.

To switch your instance to use a single-AZ configuration after upgrading, follow these steps:

Before upgrading, set the property single_az_mode under the variables object in your config.json to false (to match the existing deployment).
Upgrade to v9.2 (or later) by following the upgrade steps above.
After the upgrade completes successfully, change the single_az_mode variable to true in your config.json.
Run the apply command again with the flag --migrate-to-single-az:

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} apply --migrate-to-single-az

To check if your deployment is using a single AZ or multiple AZs, you can run the following command:

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} validate --az-status

This returns the number of AZs used in your instance.

Customize a Deployment

You can customize your existing Qrvey deployment by modifying the parameters in your config.json file and re-applying the configuration. This allows you to change various settings without needing to redeploy from scratch.

Navigate to the directory containing your config.json file.
Edit the config.json file and modify the desired parameter(s) under the "variables" object.
Save the changes.

Run the apply command to update your deployment:

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} apply

Wait for the process to complete and review the output.

Example: Change the Domain (DNS) for an Existing Instance

A common customization is setting up a custom domain for your Qrvey deployment.:

Update your config.json file:

Set the dns_zone_name property under the "variables" object to your custom domain:

{
  "account_config": {
    // ... existing account config ...
  },
  "variables": {
    // ... other variables ...
    "dns_zone_name": "qrvey.yourdomain.com"
  }
}

Apply the changes:

Run the apply command to update your deployment:

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} apply

Configure DNS:

After the apply process completes, the output includes a Load Balancer URL such as the following:
```
Load Balancer URL: abc123-1234567890.us-east-1.elb.amazonaws.com
```
You need to add a CNAME record in your DNS provider:
- Name/Host: qrvey (or your desired subdomain)
- Type: CNAME
- Value/Target: The Load Balancer URL from the output (for example, abc123-1234567890.us-east-1.elb.amazonaws.com)
- TTL: 300 (or your preferred value)
Wait for DNS propagation:

After you've added the CNAME record, it can take a few minutes to several hours for DNS changes to propagate globally, depending on your DNS provider and TTL settings.
Access your deployment:

After DNS propagation is complete, you can access your Qrvey deployment using your custom domain (for example, https://qrvey.yourdomain.com).

Note: If you're using AWS Route 53 for DNS management, when the hosted zone is in the same AWS account as your deployment, the CNAME record can be automatically created during the apply process if you have the necessary permissions configured.

Removing an Instance

To remove (destroy) a Qrvey MultiPlatform Environment instance and all associated resources:

Navigate to the directory containing your config.json file.

Run the destroy command to preview the resources that will be removed (similar to a Terraform "plan"):

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} destroy

To actually remove all resources, run the destroy command with the --approve flag:

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} destroy --approve

Warning: When the resources are removed, all data and metadata associated with the instance is permanently deleted.

Configuration Variables

This following input variables are available for AWS deployment using Terraform. Each variable can be customized to fit your deployment requirements.

Variable Name	Type	Default Value	Description
`api_key`	string	`""`	API Key for migrated instances.
`access_key_id`	string	`""`	AWS account access key.
`region`	string	`"us-east-1"`	AWS region for resource deployment.
`secret_access_key`	string	`""`	AWS account secret key.
`session_token`	string	`null`	AWS session token.
`azs`	list(string)	`null`	Availability zones for subnet creation .
`chart_name`	string	`"qrvey"`	Name of the chart to deploy.
`chart_values`	list(object)	`[]`	Chart values (name, value, type).
`create_vpc_endpoints`	bool	`true`	Whether to create VPC endpoints.
`customer_info`	object	`{}`	Required. An object containing customer information.
`deployment_id`	string	`""`	Deployment ID (for migrations).
`dns_zone_name`	string	`""`	DNS zone name.
`elasticsearch`	object	`{}`	Existing Elasticsearch engine data (host, auth_user, auth_password, cluster_name, version).
`enable_location_services`	bool	`false`	Enable location services.
`enable_trino`	bool	`false`	Deploy Trino Helm chart.
`es_config`	object	`{}`	Elasticsearch configuration (name, size, count, storage).
`globalization`	object	`{}`	Globalization settings (google_client_email, google_client_private_key, and so on).
`initial_admin_email`	string	`""`	Required. Initial admin email.
`intra_subnets_cidrs`	list(string)	`["10.110.201.0/24", "10.110.202.0/24"]`	Intra subnets.
`openai_api_key`	string	`"sk-xxxxxxxxxxxxxxxxxxxxxx"`	OpenAI API key.
`postgresql_config`	object	`{}`	PostgreSQL config (name, instance_class, version).
`private_subnets_cidrs`	list(string)	`["10.110.1.0/24", "10.110.2.0/24", "10.110.32.0/20", "10.110.48.0/20"]`	Private subnets.
`public_subnets_cidrs`	list(string)	`["10.110.101.0/24", "10.110.102.0/24"]`	Public subnets.
`qrvey_chart_version`	string	`""`	Required. Qrvey chart version.
`rabbitmq_service_internal`	bool	`true`	Use internal RabbitMQ service (true for ServiceIP, false for LoadBalancer).
`rabbitmq_replica_count`	number	3	Number of replicas for the RabbitMQ cluster. If you are upgrading from v9.1.x to 9.2.2 or later, use this flag to keep the replicas at 3.
`registry_key`	string	`""`	Required. Qrvey registry key.
`registry_user`	string	`""`	Required. Qrvey registry user.
`s3_bucket`	object	`{}`	Existing S3 bucket configuration.
`single_az_mode`	bool	`false`	If set to `true`, deploys all resources in a single Availability Zone (AZ). Set to `false` for multi-AZ deployments. Useful for controlling intra-AZ data transfer costs.
`table_hierarchy_enabled`	bool	`false`	Enable table hierarchy feature.
`trino_config`	object	`{}`	Trino configuration (name, size, count).
`use_athena_from_serverless`	bool	`false`	Use Athena from serverless.
`use_existing_vpc`	bool	`false`	Use an existing VPC.
`use_public_subnet_for_db`	bool	`false`	Use a public subnet for the database.
`vpc_cidr`	string	`"10.110.0.0/16"`	VPC CIDR block.
`vpc_details`	object	`null`	VPC details (vpc_id, public_subnets, private_subnets, intra_subnets).
`dataload_config`	object	`{}`	(Available starting version 9.2.2) Configuration for dataset loading microservices. Allows setting min/max replicas for each datarouter pod. All properties are optional.

`chart_values`

[
  {
    "name": "string",
    "value": "string",
    "type": "string"
  }
]

`customer_info`

{
  "firstname": "string",
  "lastname": "string",
  "email": "string",
  "company": "string"
}

`elasticsearch`

{
  "host": "", // optional, default
  "auth_user": "elastic", // optional, default
  "auth_password": "", // optional, default
  "cluster_name": "elasticsearch-es-internal-http.elastic-system.svc.cluster.local", // optional, default
  "version": "7.10" // optional, default
}

`es_config`

{
  "name": "elasticsearch", // optional, default
  "size": "medium", // optional, default
  "count": 1, // optional, default
  "storage": "200Gi" // optional, default
}

`size` Parameter Options

Size	node_size	JVM_MEM	POD_CPU	POD_MEM
small	m5.large	2g	1	4Gi
medium	r6i.large	4g	1	8Gi
large	r6i.xlarge	12g	2	24Gi
xlarge	r6i.2xlarge	18g	4	35Gi
2xlarge	r6i.2xlarge	24g	4	52Gi
4xlarge	r6i.4xlarge	31g	24	120Gi

`globalization`

{
  "google_client_email": "", // optional, default
  "google_client_private_key": "", // optional, default
  "google_document_id": "", // optional, default
  "google_document_sheet_title": "" // optional, default
}

`postgresql_config`

{
  "name": "postgresql", // optional, default
  "instance_class": "db.t3.medium", // optional, default
  "version": "16.3" // optional, default
}

`s3_bucket`

{
  "qrveyuserfiles": "", // optional, default
  "use_cloudfront": "true", // optional, default
  "drchunkdata": "", // optional, default
  "drdatacommons": "", // optional, default
  "drdatalake": "", // optional, default
  "config": "", // optional, default
  "basedatasets": "" // optional, default
}

`trino_config`

{
  "name": "trino", // optional, default
  "size": "small", // optional, default
  "count": 2 // optional, default
}

`vpc_details`

{
  "vpc_id": "string",
  "public_subnets": ["string"],
  "private_subnets": ["string"],
  "intra_subnets": ["string"] // optional
}

For Existing VPCs

If you are using an existing VPC and subnets, you must manually add the following tag to your private subnets and the security group named qrvey-eks-<deploymentid>-node:

karpenter.sh/discovery : qrvey-eks-<deploymentid>

This tag is required by Karpenter to create nodes. These tags are not automatically added when using existing VPCs. After adding the tags, connect to the cluster and delete any failed Helm charts as needed before proceeding with the apply step.

`dataload_config`

The dataload_config object allows you to configure resource requests/limits and autoscaling for each microservice involved in dataset loading. All properties under dataload_config are optional. If you do not specify some properties, the system will use default values, which are the ones shown in the example below. Available starting version 9.2.2.

Note: Changing these properties will directly impact the data loading process and can be used to increase or decrease performance. Increasing the maximum number of replicas may improve throughput but will also increase cloud costs, so adjust these values carefully based on your needs and budget.

{
  "dr_file_pump": {
    "resources": {
      "requests": {
        "memory": "768Mi",
        "cpu": "15m"
      },
      "limits": {
        "memory": "768Mi",
        "cpu": "1500m"
      }
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 2
    }
  },
  "dr_db_pump": {
    "resources": {
      "requests": {
        "memory": "512Mi",
        "cpu": "15m"
      },
      "limits": {
        "memory": "3072Mi",
        "cpu": "1500m"
      }
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 2
    }
  },
  "dr_join_results_pump": {
    "resources": {
      "requests": {
        "memory": "256Mi",
        "cpu": "100m"
      },
      "limits": {
        "memory": "1024Mi",
        "cpu": "1"
      }
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 10
    }
  },
  "dr_transformation": {
    "resources": {
      "requests": {
        "memory": "256Mi",
        "cpu": "100m"
      },
      "limits": {
        "memory": "2096Mi",
        "cpu": "2"
      }
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 5
    }
  },
  "dr_put_chunk_to_lake": {
    "resources": {
      "requests": {
        "memory": "256Mi",
        "cpu": "50m"
      },
      "limits": {
        "memory": "1536Mi",
        "cpu": "1"
      }
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 10
    }
  },
  "dr_put_chunk_to_dl": {
    "resources": {
      "requests": {
        "memory": "256Mi",
        "cpu": "15m"
      },
      "limits": {
        "memory": "1536Mi",
        "cpu": "1"
      }
    },
    "autoscaling": {
      "min_replicas": 1,
      "max_replicas": 10
    }
  }
}

Property Descriptions

Each top-level key (for example, dr_file_pump, dr_db_pump) represents a microservice involved in dataset loading.
resources: Specifies resource requests and limits for CPU and memory for each microservice pod.
- requests: Minimum resources guaranteed for the pod.
- limits: Maximum resources the pod can use.
autoscaling: Controls the minimum and maximum number of replicas for each microservice.
- min_replicas: Minimum number of pods to run.
- max_replicas: Maximum number of pods to run.

Troubleshooting

Helm Release Error: Another Operation in Progress

A deployment can fail with the following error:

Error: another operation (install/upgrade/rollback) is in progress

with helm_release.qrvey[0],
on k8s-cr.tf line 606, in resource "helm_release" "qrvey":
606: resource "helm_release" "qrvey" {

To resolve the issue, add the --refresh-helm flag after the apply command:

docker run --platform=linux/amd64 -v $(pwd)/config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${qrvey_version} apply --refresh-helm

Caution: This flag should be used only in these cases, as it triggers an aggressive upgrade process in which Qrvey containers are forcefully recreated.

Services or Pods Not Starting When Spot Instances Disabled

In some new AWS accounts, Spot Instances can be disabled by default. After a new deployment, if you notice that services or pods are not coming up and there are no obvious errors, it might be because of Spot Instances not being enabled in your AWS account.

To enable Spot Instances, run the following AWS CLI command:

aws iam create-service-linked-role --aws-service-name spot.amazonaws.com

After running this command, retry your deployment.

Error: creating Security Group (vpc-endpoints-sg) when using an existing VPC

If you encounter the error:

Error: creating Security Group (vpc-endpoints-sg)

The VPC endpoints probably already exist in that VPC. To avoid this issue, set the variable create_vpc_endpoints to false in your config.json file under the variables section.

If you’re using an existing VPC, your config.json should look similar to this:

"variables": {
  ...
  "create_vpc_endpoints": false,
  "azs": ["zone-id-1", "zone-id-2"],
  "use_existing_vpc": true,
  "vpc_details": {
    "vpc_id": "vpc-id",
    "public_subnets": ["subnet-id-1", "subnet-id-2"],
    "private_subnets": ["subnet-id-1", "subnet-id-2"],
    "intra_subnets": ["subnet-id-1", "subnet-id-2"]
  }
  ...
}

If you’re upgrading from a version before v9.2, make sure the variable single_az_mode is set to false.

Additional Resources

Deploy with Private EKS

Requirements​

Installation​

Upgrade to a Newer Version​

Before You Begin​

Perform the Upgrade​

Switch from Multi-AZ to Single-AZ Configuration​

Customize a Deployment​

Example: Change the Domain (DNS) for an Existing Instance​

Removing an Instance​

Configuration Variables​

chart_values​

customer_info​

elasticsearch​

es_config​

size Parameter Options​

globalization​

postgresql_config​

s3_bucket​

trino_config​

vpc_details​

For Existing VPCs​

dataload_config​

Property Descriptions​

Troubleshooting​

Helm Release Error: Another Operation in Progress​

Services or Pods Not Starting When Spot Instances Disabled​

Error: creating Security Group (vpc-endpoints-sg) when using an existing VPC​

Additional Resources​