Migration from V8 to V9 (AWS)

This document explains the process for migrating from Qrvey Platform V8 (AWS Serverless) to V9 (AWS Kubernetes). After you've started the migration process, you can use the Configure Redshift, Snowflake, and Postgres during Migration documentation to setup those connections as well.

Pre-requisites

• Install the latest version of Docker.
• The Docker image tag for the instance version you wish to install, which you can get from Customer Support.
• Obtain your <REGISTRY_USER> and <REGISTRY_KEY> values from Qrvey Support.
• Get the V9 Auto Deploy App URL from Qrvey Support.
• Ensure admin access to the AWS console where Qrvey Platform is installed. You will need an Access Key (<ACCESS_KEY_ID>) and Secret Key (<SECRET_ACCESS_KEY>) for this IAM User.
• Upgrade the Qrvey Platform to v8.8+.
• Have at least one non-IAM Qrvey Composer user account with an Administrator role. If you don’t have one, create a new user (or modify an existing one) in the Admin Center with the Administrator role. Steps are in this document.
  Note: After the upgrade, you will not be able to use IAM user credentials to access the Admin Center, so ensure you have at least one user account with Administrator privileges.
• Configure Qrvey Admin Center to use SMTP credentials for email. By default, Qrvey Platform uses AWS SES to send emails. If your instance uses SES, follow these steps to switch to SMTP:
  1. Navigate to AWS SES and generate SMTP credentials by following this document.
  2. Log in to the Admin Center and update the email settings to use an SMTP provider. Steps are here.

Minimum Permissions Required to Deploy on CloudFormation

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ECSAndFargate",
      "Effect": "Allow",
      "Action": [
        "ecs:CreateCluster",
        "ecs:CreateService",
        "ecs:RegisterTaskDefinition",
        "ecs:Describe*",
        "ecs:UpdateService",
        "ecs:DeleteService",
        "ecs:DeleteCluster",
        "ecs:DeregisterTaskDefinition",
        "ecs:ListClusters"
      ],
      "Resource": "*"
    },
    {
      "Sid": "IAMRoleAndPolicy",
      "Effect": "Allow",
      "Action": [
        "iam:CreateRole",
        "iam:PutRolePolicy",
        "iam:AttachRolePolicy",
        "iam:GetRole",
        "iam:PassRole",
        "iam:DeleteRolePolicy",
        "iam:DeleteRole"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ELBv2",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:CreateLoadBalancer",
        "elasticloadbalancing:CreateListener",
        "elasticloadbalancing:CreateTargetGroup",
        "elasticloadbalancing:ModifyLoadBalancerAttributes",
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:Describe*",
        "elasticloadbalancing:CreateRule",
        "elasticloadbalancing:DeleteListener",
        "elasticloadbalancing:DeleteRule",
        "elasticloadbalancing:DeleteTargetGroup"
      ],
      "Resource": "*"
    },
    {
      "Sid": "EC2Networking",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeVpcs",
        "ec2:DescribeSubnets",
        "ec2:DescribeSecurityGroups",
        "ec2:CreateSecurityGroup",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:DeleteSecurityGroup",
        "ec2:RevokeSecurityGroupEgress"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Logs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DeleteLogGroup",
        "logs:PutRetentionPolicy"
      ],
      "Resource": "*"
    },
    {
      "Sid": "SSM",
      "Effect": "Allow",
      "Action": [
        "ssm:PutParameter",
        "ssm:GetParameter",
        "ssm:GetParameters",
        "ssm:DescribeParameters",
        "ssm:DeleteParameter"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ECR",
      "Effect": "Allow",
      "Action": [
        "ecr:GetDownloadUrlForLayer",
        "ecr:BatchGetImage",
        "ecr:BatchCheckLayerAvailability",
        "ecr:GetAuthorizationToken"
      ],
      "Resource": "*"
    },
    {
      "Sid": "S3Access",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:CreateBucket",
        "s3:DeleteObject"
      ],
      "Resource": "*"
    },
    {
      "Sid": "DynamoDBAccess",
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeExport",
        "dynamodb:ExportTableToPointInTime",
        "dynamodb:ListTables",
        "dynamodb:DescribeTable",
        "dynamodb:UpdateContinuousBackups"
      ],
      "Resource": "*"
    },
    {
      "Sid": "CloudFormationDeployment",
      "Effect": "Allow",
      "Action": [
        "cloudformation:CreateStack",
        "cloudformation:UpdateStack",
        "cloudformation:DeleteStack",
        "cloudformation:DescribeStacks",
        "cloudformation:GetTemplate",
        "cloudformation:ValidateTemplate",
        "cloudformation:CreateUploadBucket",
        "cloudformation:GetTemplateSummary",
        "cloudformation:DescribeStackEvents",
        "cloudformation:ListStacks"
      ],
      "Resource": "*"
    }
  ]
}

Migration Steps

Step 1: Install The V9 Qrvey Platform

1. Get the CloudFormation template URL for the V9 AutoDeployApp from the Qrvey Support team.
2. Log in to your AWS Account and navigate to the CloudFormation service.
3. Create (or update) the Auto Deploy app CloudFormation stack using the URL from step 1.
4. Once the stack is created (or updated), navigate to the Outputs tab to copy and save the URL under the key EIP.
5. Call the following API to get the default config.json body. Replace <AutoDeployAppURL> with the URL from step 4.

   curl --location '<AutoDeployAppURL>/api/v1/deploy/step/migration/configJSON'
6. Set up CLI access to your AWS Account from your local machine (or remote access if using another instance).
7. Create a new folder and a new file in this folder. Name it config.json. Add the response body from step 5 as the file’s contents. Update any missing values, which you can get from Customer Support:
   • <ACCESS_KEY_ID>
   • <SECRET_ACCESS_KEY>
   • <REGISTRY_USER>
   • <REGISTRY_KEY>
   • <QRVEY_CHART_VERSION> (This is the Docker image for the instance version you want to deploy.)
8. From the CLI, log in to the Qrvey Docker Registry using the registry user and key provided by Qrvey Support.

   docker login qrvey.azurecr.io -u ${<REGISTRY_USER>} --password-stdin <<< ${<REGISTRY_KEY>}
9. Run the install commands with the desired Terraform option. Use plan to preview the deployment or apply to deploy all changes. Update the value for platform and the filename as needed.
   Note: The --platform parameter must match the machine where your Docker instance is running.

   docker run --platform=linux/amd64 -v ./config.json:/app/qrvey/config.json -it --rm qrvey.azurecr.io/qrvey-terraform-aws:${<QRVEY_CHART_VERSION>} apply
10. Once the deployment is complete, you will receive a set of credentials and URLs in the CLI. Save the following values:

• PostgresqlConnection
  Format: postgres://<POSTGRES_USER>:<POSTGRES_PASSWORD>@<POSTGRES_HOST>:<POSTGRES_PORT>/<POSTGRES_DATABASE>
• Load Balancer URL
• RabbitMQ details

---

Step 2: Deploy Migration Tool

Use Postman or Terminal to make an API request to start the creation of the Migration Utility's CloudFormation stack:

curl --location '<AutoDeployAppURL>/api/v1/deploy/step/migration/createUtilityStack' \
--header 'Content-Type: application/json' \
--data '{
  "accessKey": "<ACCESS_KEY_ID>",
  "secretKey": "<SECRET_ACCESS_KEY>",
  "pg": {
    "host": "<POSTGRES_HOST>",
    "database": "postgres",
    "port": "5432",
    "user": "<POSTGRES_USER>",
    "password": "<POSTGRES_PASSWORD>"
  },
  "rabbitmq": {
    "host": "<RABBITMQ_HOST>",       // Without any http:// or https://
    "port": "5672",                  // Port should be 5672, not 15672
    "user": "<RABBITMQ_USER>",
    "password": "<RABBITMQ_PASSWORD>"
  },
  "vpcConfig": {
    "SubnetIds": "<SUBNET_IDS>",
    "VpcId": "<VPC_ID>"
  }
}'

Once the stack creation process is complete, navigate to the Outputs section of the stack in AWS CloudFormation. Locate the Load Balancer (LB) URL in the outputs. Access the URL in a web browser to verify that the migration service's main page is displayed.

---

Step 3: Trigger Migration

Note: You may need to scale the Postgres Aurora RDS to support the data migration based on the size of data in your DynamoDB table. Contact Qrvey for more information.

Call this API to trigger the migration process and start copying the metadata:

curl --location '<Migration_Utility_URL>/api/sls-to-mp/migration' \
--header 'x-api-key: <COMPOSER_API_KEY>' \
--header 'x-api-key-public: <LICENSE_API_KEY>' \
--header 'Content-Type: application/json' \
--data '{
  "delayBeforeDisableApp": "5m",
  "omitClearTargetDbTablesBeforeMigration": false
}'

This API will return a migrationId in the response, like this:

{
  "migrationId": "<MIGRATION_ID>"
}

Save this and use it to check the progress of the migration by launching the <LOAD_BALANCER_URL> in a browser and pasting the <MIGRATION_ID> into the input box.

Once the migration status switches to READY from IN_PROGRESS, update the DNS records to point to the new Load Balancer URL. If using a custom domain for your Qrvey Platform, update it in your DNS provider. If using the QrveyApp.com URL, call this API to update the URL:

curl --location '<Migration_Utility_URL>/api/sls-to-mp/migration/<MIGRATION_ID>/restore-services' \
--header 'x-api-key: <COMPOSER_API_KEY>' \
--header 'x-api-key-public: <LICENSE_API_KEY>' \
--header 'Content-Type: application/json' \
--data '{
  "dns": {
    "name": "<DEPLOYMENT_ID>.qrveyapp.com",
    "type": "CNAME",
    "value": "<LOAD_BALANCER_URL_FROM_STEP_ONE>" 
  }
}'

---

Step 4: Test the New Environment

Test the new environment for all features before removing the old infrastructure.

---

Step 5: Cleanup V8 Resources

Install the V9 Auto Deploy app and call the following APIs to remove V8 resources:

1. Set a RETAIN policy on all S3 buckets and DynamoDB tables to prevent deletion. This should take about 10-15 minutes to process.

curl --location --request PUT '<V9_AutoDeployAppURL>/api/v1/deploy/step/migrateresourcestomp' --data 

2. Remove all V8 CloudFormation templates that are not needed in V9. This process will take about an hour. If any of the templates fail to delete, they can be manually deleted later.

curl --location --request DELETE '<V9_AutoDeployAppURL>/api/v1/deploy/step/removeserverlessstacks' --data