With Catchr, you can populate different database types:
Amazon S3
Azure SQL
Azure Blob Storage
BigQuery
Google Cloud Storage
MySQL
Postgres
Redshift
Snowflake
Connect your database to Catchr
Set up the connection between your database and Catchr via the Databases section.
Create a data stream
A data stream is the link between your connected database and Catchr. It defines where the data will be sent.
Create a job
A job sends data through the data stream and creates a table in your database.
Go to app.catchr.io and open the Database menu.
Click on + Add database.
Enter a name for your database.
This name is internal to Catchr. It will be used to identify the database within Catchr and has no impact on the actual connection.
Select the type of database you want to connect.
Fill in the connection settings.
Each database type has its required configuration. You can refer to the documentation for each database type.
Click on Save to finish the setup.
Go to app.catchr.io and open the Datastream menu.
Click on Add data stream.
Enter a name for your data stream.
This name will be used to identify the data stream within Catchr.
Select the source, i.e., where the data is coming from.
Make sure the source has been created beforehand.
Choose the destination, which is the database you previously connected.
Click Save.
Once your data stream is set up, you can create a job to populate your database.
If you’ve just created your data stream, click on + Add job at the top right of the page.
Alternatively, you can go to the datastream list and click + Add job on the corresponding row.
The job creation process includes four main steps:
Give your job a name.
This name will be used to identify the job within Catchr.
Choose the account(s) from which the data will be pulled. The accounts are available through the source you previously connected to Catchr.
You have two options:
Manually select one or more accounts, even if they come from different sources.
Automatically select all accounts from a source available in Catchr.
This option will activate all available accounts in the selected source.
Choose the fields you want to import.
The left column is the field ID, and the right column is the field name. You can rename the field name if necessary.
Then, select the partition field, which is usually the Date field.
Click Preview data.
Select a date where data is available, then click Preview.
You can pick any valid date if the platform doesn't store historical data.
This step also helps you verify that the query runs correctly.
If an error occurs, refer to the section Troubleshooting common errors at the end of this page.
Once the preview displays successfully, double-check that the data matches your expectations.
Step 4: Set up the data import schedule
This step is done in two parts:
Initial fetching: Define the period from which data should be first imported.
Automatic import: Set the frequency and the import window (how far back in time to fetch data automatically at the frequency you want).
On each run, Catchr will delete and re-write the data in the target table for the selected period.
Finally, choose the table's name as it will appear in your destination database. And click Save.
Your job will now launch the initial fetching. It will have a Running status until it is finished (depending on the period selected in the initial fetching, this can take several hours).
Once done, your job will be in Success status.
During the job creation process, you might encounter some errors during the data preview step. Here are the most common ones and how to handle them:
Error message:
"We do not have any data for the selected date. In order to proceed to the next step, you need to select a date for which you have some data."
This means the platform's API returned no data for the chosen date.
To proceed, please select a different date when data is available. This step is necessary for Catchr to verify that the request is valid.
Error message:
"This combination of fields is not possible."
This means the selected fields cannot be requested together due to API limitations.
Try simplifying your selection or removing conflicting fields.
If you encounter a different issue or need help resolving an error, please, contact us via the in-app chat or at support@catchr.io — we’re here to help!
