Intro to Datasets

A Qrvey Dataset is a collection of structured data used to create dashboards and visualizations within the platform.

Types of Datasets

Qrvey supports the following types of datasets, each with a different use-case.

Managed Datasets, Dataset Views and Base Datasets are only available in Qrvey Ultra.

• Managed Datasets: Store data inside Qrvey’s Elasticsearch-based repository.

  • Data is imported and indexed within Qrvey, making it available even if the original connection is offline.
  • Supports data transformations, scheduled imports, and full indexing.
  • Enables fast performance for dashboards, filters, aggregations, and large-scale analytics.
  • Great for any size of dataset, that does not require real-time updates.
  • Common use cases: reports, analytics, and so on.

• Dataset Views: A materialized view of a managed dataset.

  • Points to a Managed Dataset (can be in another application), without duplicating data.
  • Cannot alter or add to the source data (cannot be transformed, no joins/unions, and so on.)
  • Can superficially hide/redact data (for example, hide columns), set RLS and CLS differently, and so on.
  • Common use case: Simplifies dataset design in commingled datasets.
    • Inject tenant-level filtering / security without duplicating data.
    • Maintain one managed dataset, then create dataset views with custom-tailored columns and permissions for different applications/users.
    • When you synchronize the main dataset, all the dataset views are synced as well.

• Base Datasets: Similar to managed datasets, but data is stored as a file in S3 or Blob Storage. Only used as a data source within other Managed datasets.

  • Can be shared and referenced in other applications (see platform documentation for sharing instructions).
  • Managed Datasets (including those with geolocation, internationalization, or shared status) can be saved as Base Datasets. Saving a shared Managed Dataset as a Base Dataset does not make the Base Dataset shared by default.
    • Used to speed up joins in managed datasets: joining base datasets is faster than managed datasets.
    • Offers fewer features than managed datasets, due to limitations that arise from being stored in files.
    • Cannot create Dataset Views from Shared Base Datasets.
    • Not recommended to create base datasets from existing managed datasets unless optimizing for downstream performance; should generally be sourced from a Connection.
    • Can be joined with any other dataset type.
    • Common use cases: intended for use as join sources within managed datasets.

• Live Connect Datasets: Provide direct, real-time access to data at its source (for example, SQL, REST API).

  • Qrvey runs live queries at runtime; no data is stored or indexed in Qrvey.
  • Ideal for real-time or near real-time data needs.
  • Performance depends on the source system and may be slower for complex queries.
  • Joins, Unions, and Formulas are not supported. These must be performed in the source.
  • Limited support for advanced analytics on large tables.
  • Performance depends on the source database design, including: partitioning, indexes and keys, monitoring source for concurrency, auto-scaling, and so on.
  • Common use cases: real-time dashboards, always-fresh operational data.

• Custom Datasets: Created by exporting data from a Dataset in a Dashboard to be used for advanced analysis, pivoting, or sharing a snapshot of your data.

  • Custom Datasets are listed in the Data section with a distinct Type label: Custom and are managed separately from other datasets.
  • They are automatically kept up-to-date based on the sync schedule of the source dataset (at most every 12 hours).
  • Custom Datasets cannot be edited in the design page, and their sync schedule is always tied to the source dataset.

Summary Table

Dataset Type	Storage	Data Transformation	Real-Time	Joins	Views	Primary Use Case
Managed	Qrvey-Indexed	Yes	No	Yes	Yes	Analytics, dashboards, reports
Dataset View	Reference	No	No	No	Yes	Share data across applications without duplicating data
Base	S3/Blob File	Limited	No	Yes	No	Join optimization
Live Connect	At Source	No	Yes	No	No	Real-time dashboards
Custom	Snapshot	No	No	No	Yes	Snapshots for advanced analysis or sharing

Create a Dataset

1. Navigate to the Data Module and make sure you are in the Datasets tab.
2. Select Create New Dataset. From the dropdown menu, you can create one of the following:
   • New Managed Dataset
   • New Live Connect Dataset
   • New Base Dataset
3. Configure as desired.

Configure a Dataset

To manage a dataset, follow these steps:

1. Navigate to Data > Datasets.
2. Select the desired Dataset.
3. Configure and Manage the Dataset as needed.

You'll find configuration options under three main tabs: Design, Analyze, and Activity Log.

Design

When you've created a dataset, you can customize it as desired under the Dataset > Design tab.

Analyze

The Analyze tab provides a suite of tools for analyzing datasets. These visualizations may be added into Dashboards and embedded as widgets.

Summary View

Every dataset connected to Qrvey generates a Summary View, offering a quick overview of your data. Access it by selecting a dataset and selecting the Analyze tab.

Each column/field is displayed with relevant summary statistics. Numeric fields are visualized using bar charts, while other field types may have different visual representations.

Options in the panel's top-right menu: Download, Apply Filters, Formulas, and Buckets.

Tabular View

The Tabular View presents data in a familiar spreadsheet format.

Options in the panel's top-right menu: apply Filters, Formulas, and Buckets.

More Options:

• Filter, Sort, and Show/Hide columns
• Drag to reorder or resize columns
• Apply aggregate functions (using the three-dot menu in column headers)
• Numeric fields offer multiple aggregation options, with results displayed in the column footer.

Custom View

Custom View treats the Analyze area similar to a Dashboard to create custom visualizations.

Steps:

1. Navigate to the Analyze tab and select Custom View.
2. Select Add Chart to open the Chart Builder.
3. Select the chart's three-dot menu and use the "Size" option to adjust its size.

Options in the panel's top-right menu: Edit, Download, Duplicate, or Delete.

Metric View

The Metric View highlights key performance indicators using various visual formats.

To create a metric:

1. Navigate to the Analyze tab and select Metric View. The Chart Builder will open.
2. Choose a metric type: Indicator, Bullet, or Gauge.
3. Drag data into the fields shelf or canvas and style as needed.

Activity Log

Each Dataset gets its own Activity Log.

Browse Datasets

1. Navigate to Data > Datasets to browse all Datasets within the application.
   Each dataset will be displayed as a card, displaying the following details:
   • Name — The name assigned to the Dataset.
   • Type — Notes if it a Managed or a Live Connect Dataset.
   • Status — active (ready to create visualizations), draft (dataset is still loading), or failed (dataset is not properly loaded).
   • Last Loaded — The date that the data was loaded.
   • Data Sync — Notes if Data Sync is On or Off.
   • Data Source — The Connection used by this Dataset.
   • Records — Number of records within the dataset.
   • Columns — Number of columns within the dataset.

Mark Dataset as Favorite

To mark a Dataset as a favorite, follow these steps.

1. Navigate to Data > Datasets.
2. Find the desired dataset card.
3. Select the star in the upper-right corner.

Rename a Dataset

1. Navigate to the desired dataset card.
2. Select its name.
3. Type to change it as desired.

Share and Un-Share Datasets

By default, datasets are only accessible in the application in which they are created.

1. Navigate to Data > Datasets.
2. Find the desired Dataset and select its three-dot menu. A dropdown will appear.
3. Select either Share Data with my Organization or Unshare with my Organization. A modal will appear.
4. Confirm your choice in the modal and you will return to Data > Datasets.

Note: If unshared, any existing datasets and dataset views created from this dataset will no longer be available to users building charts and metrics. Any existing charts and metrics that have a dependency on this dataset will no longer show any data.

Delete a Dataset

1. Navigate to Data > Datasets.
2. Find the desired Dataset and select its three-dot menu. A dropdown menu will appear.
3. Select Delete. A modal will appear.
4. Select Delete to confirm your choice in the modal and you will return to Data > Datasets.

Set a Record Level Security Column

For Record Level Security (RLS), assign a security name:

1. Select the column's three-dot menu > Enable Record Level Security.
2. Enter a security name and confirm.

For more information, please see Record Level Security.

Remove RLS Column

To remove security from the dataset:

1. Select the three-dot menu.
2. Select the Record Level Security option.
3. Select Remove to reset the dataset configuration.