TABLE OF CONTENTS

• Import Run in TRP
  • Data Pull vs Data Push
  • Import Run Types: Ad Hoc vs. Scheduled Import
• Quick Import (AdHoc Run)
  • File Upload Source Type
  • Database Source Types
  • Vidyo Data Load
  • RedCap
  • TRP Database Import
• Schedule Import Runs
  • Add Schedule
  • View Schedules
  • Edit Schedule
  • Delete Schedule 
• Quick links 

Import Run in TRP

The previous article described imports as the initial step to enable data ingestion in TRP. An import tells TRP how to access data from various sources, and establishes a destination dataset to store in TRP. Once an import is set up, the next step involves initiating an 'Import Run' to retrieve the data. Different scenarios arise when accessing the data.

Data Pull vs Data Push

TRP imports can be categorized into data pull (retrieve) and data push (receive). 

Data pull is when TRP retrieves/fetches data from the data source while in data push, the data source pushes the data to TRP.

TRP performs data pulls using the import runs. While some data sources will push data to TRP e.g.

• Zoom

• MS Teams

• Webhook/Post Data

For the above data sources, data is pushed to TRP and an import run is unnecessary or applicable. Hence in this document, we will discuss data pull only.

Import Run Types: Ad Hoc vs. Scheduled Import

Imports can be run in two different ways.

1. Ad hoc, or bringing data one time on demand from the user: Quick Import

2. Periodically on a schedule: Schedule Import

Quick imports empower users to immediately access data after an import is generated and also enable users to refresh data on demand. However, for automated, user-independent data updates, import runs need to be scheduled.

Quick Import (AdHoc Run)

Quick Import runs import immediately. It also provides a shortcut to upload a new data file.

Here's a step-by-step guide on how to perform a Quick Import in TRP:

1. Navigate to the imports list.

2. Choose the dataset you want to work with.

3. In the action column, click 'Quick Import.'

The Quick Import process will provide different options depending on the source type. 

File Upload Source Type

For CSV imports, users will be prompted to upload a CSV file. This is also a shortcut to upload any new files received without going through the import process again.

Hence if a CSV import is set up for the users, they only need to use the quick import to upload any successive files.

Database Source Types

If it's a database source type, you can modify the query used to fetch the latest data. 

Here Quick Import provides a few shortcuts and helps the user to get data for different scenarios. 

Save and Run Import

This option will save any modifications made to the query. It is useful for the following situations:

• For incremental loads, change the date range of the data being fetched at each import run, e.g. if the schedule is changed to every 20 minutes from every 10 minutes, the user will want to change the date range interval to 25 minutes (leaving a 5-minute margin is generally a safe way to ensure no data is missed) to match the schedule

• For full load the user can change the date range for the whole data set, e.g data for the current year vs. data for both the current and last year 

Run Import without Saving

This option runs the import with the updated query however, it does not change the query for the import. This is useful in the following scenarios:

• Get one-time historical data for incremental loads

• Backport data for a date range where the import run had failed

Vidyo Data Load

For Vidyo CDR updates, you can execute the set query. The query is not editable.

RedCap

RedCap data is fetched without requiring any user input. 

TRP Database Import

TRP-type imports allow you to update data from other TRP datasets, the query could be modified at the time of the import run.

After the Quick Import completes, typically within a few seconds, you can click on the 'Details' button to see the number of records that were either created or updated during the import process.

Schedule Import Runs

Schedule imports are a way to run the import periodically and automatically. They’re required to update the data in TRP periodically.

This feature allows you to schedule a particular dataset’s import by scheduling when and how often an import runs. 

Add Schedule

To add a new schedule:

1. Go to the imports list. 

2. Choose your desired dataset and from the action column click on the ‘Schedule’ button. This opens up the Schedule Import popup

3. Enter the title/name of the schedule e.g. End of Shift Import.

4. Choose the scheduled time period from Hourly, Daily, Weekly, or Monthly

Hourly -  Imports updated data every hour or more frequently, allowing the most recent updates to be imported into TRP. e.g. every 20 minutes starting at 5 minutes of the hour, so the import will run three times exactly at xx:05, xx:25 and xx:55 minutes

Daily - Imports updated data at a specific time every day e.g. Daily at 9 AM Eastern Time

Weekly - Imports updated data on the chosen days of each week at a chosen period e.g. every Monday at 8 AM Eastern Time

Monthly - Imports updated data on the chosen days of each month at a chosen time e.g. first of the month at 9 AM Eastern Time

5. Choose the frequency from the dropdown, or how often the import should run.

6. Click on ‘Add Schedule’ to enter and save this in the system. 

View Schedules

1. Access Schedule List

Find a list of all saved schedules under "Current Schedules."

2. Navigate to Scheduled Tasks

To explore all scheduled imports, go to “Scheduled Tasks” in the Imports and Integrations section.

This will display a list of all scheduled imports. 

Clicking on the “Details” button under the Action column will direct users to a detailed view where user can also perform the following actions: 

1- Edit Scheduled Imports

2- View logs 

3- Delete scheduled import

Edit Schedule

There are two ways to edit scheduled imports. 

1. Instant Edit After Creation

After creating an import, use the small blue pencil button for instant editing.

1. Edit from Scheduled Tasks

Alternatively, go to “Scheduled Tasks” and select the desired schedule. Click the “Details” button and then hit “Edit.”

Click save once the edit is completed. 

Delete Schedule 

There are two ways to delete scheduled imports. 

1. Instant Deletion After Creation

Right after creating an import, use the red button for instant deletion if a schedule is no longer needed.

2. Delete from Scheduled Tasks

Go to “Scheduled Tasks,” select the schedule, click “Details,” and then hit “Delete” to proceed with deletion.

Quick links 

Introduction to Imports in TRP

Available Imports Sources in TRP