Salesforce to BigQuery - Data Transfer

[hsenoussi]

Hello,

Have anyone tested the new data transfer from Salesforce ? https://cloud.google.com/bigquery/docs/salesforce-transfer

Something that isn't clear in the documentation, does it concern Salesforce Sales Cloud ? 

What are your first feedbacks ?

Best regards,

[maricarp · Accepted solution]

Hello @hsenoussi ,

For clarifications about BigQuery Data Transfer for Salesforce, I recommend contacting dts-preview-support@google.com to get support or provide feedback as noted here in the documentation.

Hope this helps.

[Snoshone07]

Hi @hsenoussi yes, the new Salesforce data transfer feature is specifically designed to extract data from Salesforce and load it directly into BigQuery for seamless analysis. Here’s a breakdown with clarifications and feedback based on my experience:

1. Does it Support Salesforce Sales Cloud?

Absolutely. The Salesforce Data Transfer feature in BigQuery supports Salesforce Sales Cloud, as well as other products like Service Cloud. It handles data from standard Salesforce objects (e.g., Accounts, Leads, Opportunities) and can also work with custom objects, provided your Salesforce account has API access enabled.

2. Key Features of the Integration

Ease of Use: Setting up a scheduled transfer from Salesforce to BigQuery is straightforward and doesn’t require complex ETL pipelines.

• Custom Object Support: You can include custom objects and fields, but you may need to configure these manually during setup.
• Incremental Updates: After the initial data load, the integration supports incremental updates, ensuring only new or updated records are imported.

3. Feedback from Testing

Here are some insights based on initial testing:

• Setup:
  The setup is intuitive if you have the right permissions in both Salesforce and BigQuery. You’ll need a Salesforce user account with API access and the ability to create connected apps for OAuth authentication.

• Data Volume:
  For Salesforce instances with millions of records, the initial data load can take some time. However, subsequent incremental updates are much faster.

• Limitations:

  • Certain Salesforce fields, like formula fields or polymorphic fields, might require additional configuration or won’t map directly to BigQuery.
  • Salesforce’s API limits (depending on your edition) could restrict the volume and frequency of data transfers.

• Documentation:
  While the documentation provides a solid overview, additional details on managing custom objects and addressing API limits would be helpful.

4. External connectors as an Alternative

If the native integration doesn’t fully meet your needs, Windsor.ai is an excellent alternative for connecting Salesforce data to BigQuery.

Why Consider Windsor.ai?

• Customizable: Easily include standard and custom objects with advanced field mapping options.
• Optimized API Usage: Efficiently handles API limits by batching requests and offering detailed logs for transparency.
• Multi-Destination Support: In addition to BigQuery, Windsor.ai integrates with platforms like Snowflake, Azure, and Power BI.
• No-Code Setup: The platform allows you to configure pipelines without any coding experience.

Getting Started:

1. Visit Windsor.ai
2. Set up the Salesforce connector and authenticate using your Salesforce credentials.
3. Select BigQuery as the data destination and configure the sync frequency to match your needs.

5. Final Thoughts

The native Salesforce Data Transfer feature for BigQuery is a powerful solution for businesses already using both platforms and seeking a straightforward way to automate data movement. However, for advanced customization, more flexibility, or multi-destination capabilities, a third-party tool can complement or enhance your setup.

Good luck!

[prakhar-delhive]

@Snoshone07 I just did a setup of Salesforce Transfer to BQ, but i dont see an option to configure incremental data load. Currently what is happening, my whole data is getting refreshed from Salesforce impacting the deleted records also vanish away. Could you suggest a way out here ?