Intro to Datasets

A Qrvey Dataset is a collection of structured data used to create dashboards and visualizations within the platform.

Types of Datasets

There are four types of datasets, each with a different use-case.

Managed Datasets, Dataset Views & Base Datasets are only available in Qrvey Ultra.

• Managed Datasets: Store data inside Qrvey’s Elasticsearch-based repository.

  • Data is imported and indexed within Qrvey, making it available even if the original connection is offline.
  • Supports data transformations, scheduled imports, and full indexing.
  • Enables fast performance for dashboards, filters, aggregations, and large-scale analytics.
  • Great for any size of dataset, that does not require real-time updates.
  • Common use cases: reports, analytics, etc.

• Dataset Views: A materialized view of a managed dataset.

  • Points to a Managed Dataset (can be in another application), without duplicating data.
  • Cannot alter or add to the source data (cannot be transformed, no joins/unions, etc.)
  • Can superficially hide/redact data (e.g., hide columns), set RLS & CLS differently, etc.
  • Common use case: Simplifies dataset design in commingled datasets.
    • Inject tenant-level filtering / security without duplicating data.
    • Maintain one managed dataset, then create dataset views with custom-tailored columns & permissions for different applications/users.
    • When you synchronize the main dataset, all the dataset views are synced as well.

• Base Datasets: Similar to managed datasets, but data is stored as a file in S3. Only used as a data source within other Managed datasets.

  • Used to speed up joins in managed datasets: joining base datasets is faster than managed datasets.
  • Offers fewer features than managed datasets, due to limitations that arise from being stored in S3 files.
  • Not recommended to create base datasets from existing managed datasets, should be sourced from a Connection.
  • Can be joined with any other dataset type.
  • Common use cases: Cannot be used directly for visualizations; intended for use as join sources within managed datasets.

• Live Connect Datasets: Provide direct, real-time access to data at its source (e.g., SQL, REST API).

  • Qrvey runs live queries at runtime; no data is stored or indexed in Qrvey.
  • Ideal for real-time or near real-time data needs.
  • Performance depends on the source system and may be slower for complex queries.
  • Joins, Unions, and Formulas are not supported. These must be performed in the source.
  • Limited support for advanced analytics on large tables.
  • Performance depends on the source database design, including: partitioning, indexes and keys, monitoring source for concurrency, auto-scaling, etc.
  • Common use cases: real-time dashboards, always-fresh operational data.

Create a Dataset

1. Navigate to the Data Module and make sure you are in the Datasets tab.
2. Click Create New Dataset. A dropdown will appear. Choose to create a:
   • New Managed Dataset
   • New Live Connect Dataset
   • New Base Dataset
3. Configure as Desired.

Configure a Dataset

To manage a dataset, follow these steps:

1. Navigate to Data > Datasets.
2. Click on the desired Dataset.
3. Configure and Manage the Dataset as needed.

You'll find configuration options under three main tabs: Design, Analyze, and Activity Log.

Design

Once you've created a dataset, you can customize it as desired under the Dataset > Design tab.

Analyze

The Analyze tab provides a suite of tools for analyzing datasets. These visualizations may be added into Dashboards and embedded as widgets.

Summary View

Every dataset connected to Qrvey generates a Summary View, offering a quick overview of your data. Access it by selecting a dataset and clicking the Analyze tab.

Each column/field is displayed with relevant summary statistics. Numeric fields are visualized using bar charts, while other field types may have different visual representations.

Options in the panel's top-right menu: Download, Apply Filters, Formulas, and Buckets.

Tabular View

The Tabular View presents data in a familiar spreadsheet format.

Options in the panel's top-right menu: apply Filters, Formulas, and Buckets.

More Options:

• Filter, Sort, and Show/Hide columns
• Drag to reorder or resize columns
• Apply aggregate functions (via the three-dot menu in column headers)
• Numeric fields offer multiple aggregation options, with results displayed in the column footer.

Custom View

Custom View treats the Analyze area similar to a Dashboard to create custom visualizations.

Steps:

1. Navigate to the Analyze tab and select Custom View.
2. Click Add Chart to open the Chart Builder.
3. Click on the chart's 3-dot menu and use the "Size" option to adjust its size.

Options in the panel's top-right menu: Edit, Download, Duplicate, or Delete.

Metric View

The Metric View highlights key performance indicators using various visual formats.

To create a metric:

1. Navigate to the Analyze tab and select Metric View. The [Chart Builder](../07-Charts/the-chart-builder.md will open.
2. Choose a metric type: Indicator, Bullet, or Gauge.
3. Drag data into the fields shelf or canvas and style as needed.

Activity Log

Each Dataset gets its own Activity Log.

Browse Datasets

1. Navigate to Data > Datasets to browse all Datasets within the application.
   Each dataset will be displayed as a card, displaying the following details:
   • Name — The name assigned to the Dataset.
   • Type — Notes if it a Managed or a Live Connect Dataset.
   • Status — active (ready to create visualizations), draft (dataset is still loading), or failed (dataset is not properly loaded).
   • Last Loaded — The date that the data was loaded.
   • Data Sync — Notes if Data Sync is On or Off.
   • Data Source — The Connection used by this Dataset.
   • Records — Number of records within the dataset.
   • Columns — Number of columns within the dataset.

Mark Dataset as Favorite

To mark a Dataset as a favorite, follow these steps.

1. Navigate to Data > Datasets.
2. Find the desired dataset card.
3. Click the star in the upper-right corner.

Rename a Dataset

1. Navigate to the desired dataset card.
2. Click on its name.
3. Type to change it as desired.

Share and Un-Share Datasets

By default, datasets are only accessible in the application in which they are created.

1. Navigate to Data > Datasets.
2. Find the desired Dataset and click its three-dot menu. A dropdown will appear.
3. Click either Share Data with my Organization or Unshare with my Organization. A modal will appear.
4. Click to confirm your choice in the modal and you will return to Data > Datasets.

Note: If unshared, any existing datasets and/or dataset views created from this dataset will no longer be available to users building charts and metrics. Any existing charts and metrics that have a dependency on this dataset will no longer show any data.

Delete a Dataset

1. Navigate to Data > Datasets.
2. Find the desired Dataset and click its three-dot menu. A dropdown menu will appear.
3. Click Delete. A modal will appear.
4. Click Delete to confirm your choice in the modal and you will return to Data > Datasets.

Set a Record Level Security Column

For Record Level Security (RLS), assign a security name:

1. Click the column's three-dot menu > Enable Record Level Security.
2. Enter a security name and confirm.

For more information, please see Record Level Security.

Remove RLS Column

To remove security from the dataset:

1. Click the three-dot menu.
2. Select the Record Level Security option.
3. click Remove to reset the dataset configuration.