search shadow

Menu

Articles in this section

See more

Database Sync for HubSpot Guide Follow

Index

  1. Introduction
  2. Setup Preparation - MS SQL
  3. Setup Preparation - MySQL
  4. Setup Preparation - BigQuery
  5. Setup Preparation - PostgreSQL
  6. Setup Preparation - Redshift
  7. Account Creation and Setup
  8. Activation
  9. Opt-In Tables
  10. Temporary Tables
  11. Table Structure
  12. How to Uninstall the app
  13. Beta HubSpot Apis
  14. Data Sync
  15. Data Security

Introduction

The Datawarehouse.io Database Sync for HubSpot App facilitates a connection between HubSpot and your database of choosing. This sync is one-directional from HubSpot to your database.

This app currently supports the following database formats with more coming in the future.

  • Azure SQL Database
  • Azure Synapse Analytics
  • Azure Data Warehouse
  • Standalone Microsoft SQL Server
  • AWS RDS MS SQL
  • Google Cloud SQL for SQL Server
  • Google Big Query
  • PostgreSQL
  • MySQL

Setup Preparation - MS SQL Variations

In order to complete the app installation process you will need the following information prepared ahead of time.

  1. Your Server Address
  2. Your Database Name 
  3. A Database Username (This must not be the server admin user It must specifically be a database user for the database you create for us to sync data to)
  4. Password for the above Username

The following permissions must be granted to the Database Username used to run this application. Without these permissions the application will not function correctly.

db_datareader
db_datawriter
db_ddladmin
GRANT VIEW SERVER STATE TO <user_name>

 

If you have the above information ready then proceed to Account Creation and Setup. Please note that if the database you provide to connect to our service is not sufficiently fast, the sync may be delayed. Support can work with you to define the right level of database power if what you currently have is insufficient.

 

Optional Preparation

Depending on how you have set up your security you may also need to whitelist our App Services' IP address range. Depending on if you Select EU or US app location, the ranges for the IP addresses differ. The IP ranges you may need to whitelist are downloadable as a word doc below.

 

EU IP Range

NA IP Range

 

You may need to open port 1433 to connect to our App Service depending on your hosting method and security settings.

If you require the use of a custom port other than port 1433 to connect to Database Sync, you will need to enter both the server address along with the port you will be using in the following manner: <serveraddress>,<port>. Just be sure to remove the "<>".

 

Setup Preparation - MySQL

In order to complete the app installation process you will need the following information prepared ahead of time.

  1. Your Server Address
  2. Your Database/Schema Name
  3. Database Username 
  4. Password for the above Username

You must create a schema for your HubSpot data. For example you can run the following query.

 

CREATE SCHEMA `hsc_db`;

 

You must also create a user for your database. An example is shown below. Do not use the example when creating your user.

 

CREATE USER 'hsc_user'@'%' IDENTIFIED BY 'SomeStrongPassword1234';

 

The following permissions must be granted to the Database Username used to run this application. Without these permissions the application will not function correctly.

 

GRANT ALL PRIVILEGES ON hsc_db.* TO 'hsc_user'@'%';

 

If you have the above information ready then proceed to Account Creation and Setup. Please note that if the database you provide to connect to our service is not sufficiently fast, the sync may be delayed. Support can work with you to define the right level of database power if what you currently have is insufficient.

 

Optional Preparation

Depending on how you have set up your security you may also need to whitelist our App Services' IP address range. Depending on if you Select EU or US app location, the ranges for the IP addresses differ. The IP ranges you may need to whitelist are downloadable as a word doc below.

 

EU IP Range

NA IP Range

 

You may need to open port 1433 to connect to our App Service depending on your hosting method and security settings.

If you require the use of a custom port other than port 1433 to connect to Database Sync, you will need to enter both the server address along with the port you will be using in the following manner: <serveraddress>,<port>. Just be sure to remove the "<>".

 

 

Setup Preparation - BigQuery

When naming your Dataset during setup and authorization Dataset IDs/names must be alphanumeric (plus underscores) and must be at most 1024 characters long. If you use other characters the sync will not be able to proceed.

We utilize Google SSO and the Oath tokens we acquire through it in order to acquire the API scopes necessary for our app to function. As such the only preparation you need to undertake is to verify your google account has the ability to grant access to the following scopes when installing the app.

 

API Scope User-facing description
  ../auth/userinfo.email See your primary Google Account email address
BigQuery API .../auth/bigquery View and manage your data in Google BigQuery and see the email address for your Google Account
BigQuery API .../auth/cloud-platform.read-only View your data across Google Cloud services and see the email address of your Google Account

 

If your Oath tokens in Google Big Query are set to expire our sync will stop once the token expires. You will need to perform one of the two following steps to resolve this otherwise your sync will be halted:

A. Set "Database Sync for HubSpot by Datawarehouse.io" as a Trusted App and then exempt Trusted Apps from reauthentication.

B. Change the token expiration settings to never require reauthentication for any app.

See https://support.google.com/a/answer/9368756 for more details on this.

This is the only supported resolution by google for handling syncs to big query. Otherwise its not possible for any third party to sync data to big query without the token expiring.

 

When granting the app access to BigQuery. You must select the following options in order for the sync to work. This window will appear during the app install process.

 

 

Setup Preparation - PostgreSQL Variations

In order to complete the app installation process you will need the following information prepared ahead of time.

  1. Server address and port number. If your port number differs from 5432, your address must be entered in this format "servername,port" or "ip,port"
  2. Your Database Name 
  3. Database Username 
  4. Password for the above Username
  5. Schema name. The default we use is set to "public" if you do not or cannot provide this.

The following permissions and roles must be granted to the Database Username used to run this application. Without these roles the application will not function correctly.

 

Permissions

GRANT ALL ON DATABASE "databasename" TO "username"
GRANT ALL ON SCHEMA "schemaname" TO "username" (use a schema name you define or set it to "public".

Roles

pg_read_all_data

pg_write_all_data

 

If you have the above information ready then proceed to Account Creation and Setup. Please note that if the database you provide to connect to our service is not sufficiently fast, the sync may be delayed. Support can work with you to define the right level of database power if what you currently have is insufficient.

 

Optional Preparation

 

On the setup screen for database sync, you are given the option to define whether table and column names should be in all lowercase or PascalCase. Either option will work. This choice is personal preference. 

 

 

Depending on how you have set up your security you may also need to whitelist our App Services' IP address range. Depending on if you Select EU or US app location, the ranges for the IP addresses differ. The IP ranges you may need to whitelist are downloadable as a word doc below.

 

EU IP Range

NA IP Range

 

If you require the use of a custom port other than port 5432 to connect to Database Sync, you will need to enter both the server address along with the port you will be using in the following manner: "<serveraddress>,<port>". Just be sure to remove the "<>".

 

Setup Preparation - AWS Redshift

In order to complete the app installation process you will need the following information prepared ahead of time.

  1. Server Endpoint 
  2. Database Name (This is set to "dev" by default)
  3. Database Username 
  4. Password for the above Username
  5. Schema name. The default we use is set to "public" if you do not or cannot provide this.

Your Redshift Server "Endpoint" can be found in your Workgroup configuration page under general information.

Additionally, your Redshift server must be Publicly accessible. This can be defined on the workgroup configuration screen in AWS under Data Access > Network and security.

 

The following permissions and roles must be granted to the Database Username used to run this application. Without these roles the application will not function correctly.

 

Permissions

GRANT ALL ON DATABASE "databasename" TO "username"
GRANT ALL ON SCHEMA "schemaname" TO "username" (use a schema name you define or set it to "public".

Roles

GRANT ROLE "sys;dba" TO "username"

 

An example of the query necessary in order to prepare your redshift instance can be found below. This query can be run in the Redshift query editor.

You would run each line in ascending order to create the database user, password, schema, and assign permissions.

 

If you have the above information ready then proceed to Account Creation and Setup. Please note that if the database you provide to connect to our service is not sufficiently fast, the sync may be delayed. Support can work with you to define the right level of database power if what you currently have is insufficient.

 

During setup you will be expected to enter the following information.

 

 

Once you have entered the information into the connection details section shown above, you are ready to start your trial of Database Sync.

 

Optional Preparation

Depending on how you have set up your security you may also need to whitelist our App Services' IP address range. Depending on if you Select EU or US app location, the ranges for the IP addresses differ. The IP ranges you may need to whitelist are downloadable as a word doc below.

 

EU IP Range

NA IP Range

 

Account Creation and Setup

1. Create an account at https://datawarehouse.io/account/

2. To connect to your app of choice visit the following link and select the app you wish you use. (https://datawarehouse.io/products/).

3. Select the Refresh Rate that you wish to use.

4. Click the get started link on the app product page.

5. You will be re-directed to an authentication page. Please click the Authorize button.

6. Login to HubSpot if you are not already logged in and select the HubSpot account you wish to connect too.

7. Authorize the app to connect to your HubSpot portal by clicking the Connect App button.

8. You then will be re-directed to the app setup screen shown below. Select your Database Type.

 

mceclip0.png

 

9. After selecting your database type please enter your credentials, or depending on your selection authenticate with your platform of choice, so our app can communicate with your database. Then click the continue button to proceed to the next step.

 

mceclip1.png

 

10. You are then re-directed to the checkout screen. Please fill out all required information and start your trial in order to activate your service.  The service will automatically activate upon completion of checkout.

 

Activation

Once you have successfully signed up and completed the checkout process, we will automatically activate your app. Upon Activation the app will begin to create all of the necessary tables required in order to sync your HubSpot data.

 

You can visit https://datawarehouse.io/account/my-app-information-item/ and Select "Database Sync" to test and edit your connection.

 

Please give the service 24 hours to complete the initial population of data into your database. After that you are good to go and the sync will occur at the interval you selected.

 

Opt-In Tables

The following tables are opt-in as they are advanced and are large data sets and can take up a significant amount of storage and API Calls. Submit a support request with your database name or email help@datawarehouse.io to ask for these to be enabled. There is no additional cost to enable these tables. 

 

ContactWebEvents

CustomEvent

ContactSubscriptionType

ChatflowThread

ChatflowMessage

TicketPropertiesHistory

DealPropertiesHistory

ContactPropetiesHistory

CompanyPropertiesHistory

EngagementsHistory

 

Temporary Tables

Your database may have tables with the prefix tmp. or _tmpreceive or _tmpupload

These exist to improve the data syncing process for the associations tables. Please do not use them in any reports or integrations.

 

Table Structure

The data tables and schemas that our app creates are not designed to be altered. If you must alter the tables that our app creates, understand that it may affect your sync speed.

 

Note for SQL Express users

SQL Express does not open TCP/IP connections by default. In order for our app to work if you are using SQL Express, you will need to enable TCP/IP Connections to the server.

 

How to Uninstall the app

*Please note this does not cancel your subscription. To do that please visit https://datawarehouse.io/account/view-subscription

1.  Open up the HubSpot Portal that you have installed the app in. In the top right section of HubSpot click on the house icon.

2. A menu will drop down. Click on the Connected apps link.

 

mceclip0.png

3.  Then click on the actions button for the MS SQL Connector. And click uninstall.

 

Beta HubSpot Apis

This app utilizes the following beta APIs

ContactWebEvents data table that contains web analytics history for Contacts is powered by the beta API listed below. https://developers.hubspot.com/docs/api/events/web-analytics

Feedback Submissions data table that contains Feedback Submission data is powered by the beta API listed below.

https://developers.hubspot.com/docs/api/crm/feedback-submissions 

ChatflowThread and ChatflowMessage data tables contain Chatflows data and is powered by the beta API listed below.

https://developers.hubspot.com/docs/api/conversations/conversations 

The Lists tables are powered by the beta Lists API listed below.

https://developers.hubspot.com/docs/api/crm/lists 

More information about beta APIs can be found in this supporting documentation.

 

Data Sync

Data syncing will be done at intervals determined by your plan.

Please allow at least 24-48 hours from activation for the initial database to be populated. Once it is built it will refresh at the rate determined by your plan.

 

Data Security

  • Only you will have access to your database. The permissions to connect are encrypted and only the app will be able to use them. They are not accessible anywhere else.
  • App authentication for HubSpot is done using OAuth 2.0 protocol as per the best practice in industry.
  • This application is GDPR Compliant. If your billing country is located in the EU the app will be hosted and processed in Microsoft Azure's EU West Region.
  • If there is information missing from a property it could be because that fields is improperly configured in HubSpot.

Related articles