Facebook Marketing
This page guides you through the process of setting up the Facebook Marketing source connector.
Prerequisites
For Airbyte Cloud
If you are not the owner/admin of the Ad account, you must be granted permissions to access the Ad account by an admin.
For Airbyte Open Source
A Facebook app with the Marketing API enabled and the following permissions:
Setup guide
(For Airbyte Open Source) Generate an access token and request a rate limit increase
To set up Facebook Marketing as a source in Airbyte Open Source, you will first need to create a Facebook app and generate a Marketing API access token. You will then need to request a rate limit increase from Facebook. The following steps will guide you through this process:
-
Navigate to Meta for Developers and follow the steps provided in the Facebook documentation to create a Facebook app.
-
While creating the app, when you are prompted for "What do you want your app to do?", select Other. You will also need to set the app type to Business when prompted.
-
From your App’s dashboard, set up the Marketing API.
-
Generate a Marketing API access token: From your App’s Dashboard, click Marketing API --> Tools. Select all the available token permissions (
ads_management
,ads_read
,read_insights
,business_management
) and click Get token. Copy the generated token for later use. -
Request a rate limit increase: Facebook heavily throttles API tokens generated from Facebook apps with the default Standard Access tier, making it infeasible to use the token for syncs with Airbyte. You'll need to request an upgrade to Advanced Access for your app on the following permissions:
- Ads Management Standard Access
- ads_read
- Ads_management
See the Facebook documentation on Authorization to request Advanced Access to the relevant permissions.
You can use the Access Token Tool at any time to view your existing access tokens, including their assigned permissions and lifecycles.
Set up Facebook Marketing as a source in Airbyte
-
Log in to your Airbyte Cloud account, or navigate to your Airbyte Open Source dashboard.
-
In the left navigation bar, click Sources. In the top-right corner, click + New source.
-
Find and select Facebook Marketing from the list of available sources.
-
For Source name, enter a name for your Facebook Marketing connector.
-
To authenticate the connection:
For Airbyte Cloud: Click Authenticate your account to authorize your Facebook account. Make sure you are logged into the right account, as Airbyte will authenticate the account you are currently logged in to.
For Airbyte Open Source: In the Access Token field, enter the access token you generated with your Facebook app.
Facebook Marketing Source Settings
-
For Account ID(s), enter one or multiple comma-separated Facebook Ad Account ID Numbers to use when pulling data from the Facebook Marketing API. To find this ID, open your Meta Ads Manager. The Ad Account ID number is in the Account dropdown menu or in your browser's address bar. Refer to the Facebook docs for more information.
-
(Optional) For Start Date, use the provided datepicker, or enter the date programmatically in the
YYYY-MM-DDTHH:mm:ssZ
format. If not set then all data will be replicated for usual streams and only last 2 years for insight streams.warningInsight tables are only able to pull data from the last 37 months. If you are syncing insight tables and your start date is older than 37 months, your sync will fail.
-
(Optional) For End Date, use the provided datepicker, or enter the date programmatically in the
YYYY-MM-DDTHH:mm:ssZ
format. This is the date until which you'd like to replicate data for all Incremental streams. All data generated between the start date and this end date will be replicated. Not setting this option will result in always syncing the latest data. -
(Optional) Multiselect the Campaign Statuses to include data from Campaigns for particular statuses.
-
(Optional) Multiselect the AdSet Statuses to include data from AdSets for particular statuses.
-
(Optional) Multiselect the Ad Statuses to include data from Ads for particular statuses.
-
(Optional) Toggle the Fetch Thumbnail Images button to fetch the
thumbnail_url
and store the result inthumbnail_data_url
for each Ad Creative. -
(Optional) In the Custom Insights section, you may provide a list of ad statistics entries. Each entry should have a unique name and can contain fields, breakdowns or action_breakdowns. Fields refer to the different data points you can collect from an ad, while breakdowns and action_breakdowns let you segment this data for more detailed insights. Click on Add to create a new entry in this list.
noteTo retrieve specific fields from Facebook Ads Insights combined with other breakdowns, you can choose which fields and breakdowns to sync. However, please note that not all fields can be requested, and many are only functional when combined with specific other fields. For example, the breakdown
app_id
is only supported with thetotal_postbacks
field. For more information on the breakdown limitations, refer to the Facebook documentation.To configure Custom Insights:
-
For Name, enter a name for the insight. This will be used as the Airbyte stream name.
-
(Optional) For Level, enter the level of granularity for the data you want to pull from the Facebook Marketing API (
account
,ad
,adset
,campaign
). Set toad
by default. -
(Optional) For Fields, use the dropdown list to select the fields you want to pull from the Facebook Marketing API.
-
(Optional) For Breakdowns, use the dropdown list to select the breakdowns you want to configure.
-
(Optional) For Action Breakdowns, use the dropdown list to select the action breakdowns you want to configure.
-
(Optional) For Action Report Time, enter the action report time you want to configure. This value determines the timing used to report action statistics. For example, if a user sees an ad on Jan 1st but converts on Jan 2nd, this value will determine how the action is reported.
impression
: Actions are attributed to the time the ad was viewed (Jan 1st).conversion
: Actions are attributed to the time the action was taken (Jan 2nd).mixed
: Click-through actions are attributed to the time the ad was viewed (Jan 1st), and view-through actions are attributed to the time the action was taken (Jan 2nd).
-
(Optional) For Time Increment, you may provide a value in days by which to aggregate statistics. The sync will be chunked into intervals of this size. For example, if you set this value to 7, the sync will be chunked into 7-day intervals. The default value is 1 day.
-
(Optional) For Start Date, enter the date in the
YYYY-MM-DDTHH:mm:ssZ
format. The data added on and after this date will be replicated. If this field is left blank, Airbyte will replicate all data. -
(Optional) For End Date, enter the date in the
YYYY-MM-DDTHH:mm:ssZ
format. The data added on and before this date will be replicated. If this field is left blank, Airbyte will replicate the latest data. -
(Optional) For Custom Insights Lookback Window, you may set a window in days to revisit data during syncing to capture updated conversion data from the API. Facebook allows for attribution windows of up to 28 days, during which time a conversion can be attributed to an ad. If you have set a custom attribution window in your Facebook account, please set the same value here. Otherwise, you may leave it at the default value of 28. For more information on action attributions, please refer to the Meta Help Center.
warningAdditional data streams for your Facebook Marketing connector are dynamically generated according to the Custom Insights you specify. If you have an existing Facebook Marketing source and you decide to update or remove some of your Custom Insights, you must also adjust the connections that sync to these streams. Specifically, you should either disable these connections or refresh the source schema associated with them to reflect the changes.
-
-
(Optional) For Page Size of Requests, you can specify the number of records per page for paginated responses. Most users do not need to set this field unless specific issues arise or there are unique use cases that require tuning the connector's settings. The default value is set to retrieve 100 records per page.
-
(Optional) For Insights Window Lookback, you may set a window in days to revisit data during syncing to capture updated conversion data from the API. Facebook allows for attribution windows of up to 28 days, during which time a conversion can be attributed to an ad. If you have set a custom attribution window in your Facebook account, please set the same value here. Otherwise, you may leave it at the default value of 28. For more information on action attributions, please refer to the Meta Help Center.
-
(Optional) For Insights Job Timeout, you may set a custom value in range from 10 to 60. It establishes the maximum amount of time (in minutes) of waiting for the report job to complete.
-
Click Set up source and wait for the tests to complete.
Supported sync modes
The Facebook Marketing source connector supports the following sync modes:
- Full Refresh - Overwrite
- Full Refresh - Append
- Incremental Sync - Append (except for the AdCreatives and AdAccount tables)
- Incremental Sync - Append + Deduped (except for the AdCreatives and AdAccount tables)
Supported streams
The rule
field may not be synced for all records because it caused the error message Please reduce the amount of data...
.
Airbyte also supports the following Prebuilt Facebook Ad Insights Reports:
Stream | Breakdowns | Action Breakdowns |
---|---|---|
Ad Insights Action Carousel Card | --- | action_carousel_card_id , action_carousel_card_name |
Ad Insights Action Conversion Device | device_platform | action_type |
Ad Insights Action Product ID | product_id | --- |
Ad Insights Action Reaction | --- | action_reaction |
Ad Insights Action Video Sound | --- | action_video_sound |
Ad Insights Action Video Type | --- | action_video_type |
Ad Insights Action Type | --- | action_type |
Ad Insights Age And Gender | age , gender | action_type , action_target_id , action_destination |
Ad Insights Delivery Device | device_platform | action_type |
Ad Insights Delivery Platform | publisher_platform | action_type |
Ad Insights Delivery Platform And Device Platform | publisher_platform , device_platform | action_type |
Ad Insights Demographics Age | age | action_type |
Ad Insights Demographics Country | country | action_type |
Ad Insights Demographics DMA Region | dma | action_type |
Ad Insights Demographics Gender | gender | action_type |
Ad Insights DMA | dma | action_type , action_target_id , action_destination |
Ad Insights Country | country | action_type , action_target_id , action_destination |
Ad Insights Platform And Device | publisher_platform , platform_position , impression_device | action_type |
Ad Insights Region | region | action_type , action_target_id , action_destination |
You can segment the Ad Insights table into parts based on the following information. Each part will be synced as a separate table if normalization is enabled:
- Country
- DMA (Designated Market Area)
- Gender & Age
- Platform & Device
- Region
For more information, see the Facebook Insights API documentation.
Facebook Marketing Attribution Reporting
The Facebook Marketing connector uses the lookback_window
parameter to repeatedly read data from the last <lookback_window>
days during an Incremental sync. This means some data will be synced twice (or possibly more often) despite the cursor value being up to date, in order to capture updated ads conversion data from Facebook. You can change this date window by adjusting the lookback_window
parameter when setting up the source, up to a maximum of 28 days. Smaller values will result in fewer duplicates, while larger values provide more accurate results. For a deeper understanding of the purpose and role of the attribution window, refer to this Meta article.
Data type mapping
Integration Type | Airbyte Type |
---|---|
string | string |
number | number |
array | array |
object | object |
Reference
Config fields reference
Changelog
Version | Date | Pull Request | Subject |
---|---|---|---|
1.4.0 | 2024-02-08 | 32449 | Replace "Include Deleted Campaigns, Ads, and AdSets" option in configuration with specific statuses selection per stream |
1.3.1 | 2024-02-05 | 34845 | Add missing fields to schemas |
1.3.0 | 2024-01-09 | 33538 | Updated the Ad Account ID(s) property to support multiple IDs |
1.2.3 | 2024-01-04 | 33934 | Make ready for airbyte-lib |
1.2.2 | 2024-01-02 | 33828 | Add insights job timeout to be an option, so a user can specify their own value |
1.2.1 | 2023-11-22 | 32731 | Removed validation that blocked personal ad accounts during check |
1.2.0 | 2023-10-31 | 31999 | Extend the AdCreatives stream schema |
1.1.17 | 2023-10-19 | 31599 | Base image migration: remove Dockerfile and use the python-connector-base image |
1.1.16 | 2023-10-11 | 31284 | Fix error occurring when trying to access the funding_source_details field of the AdAccount stream |
1.1.15 | 2023-10-06 | 31132 | Fix permission error for AdAccount stream |
1.1.14 | 2023-09-26 | 30758 | Exception should not be raises if a stream is not found |
1.1.13 | 2023-09-22 | 30706 | Performance testing - include socat binary in docker image |
1.1.12 | 2023-09-22 | 30655 | Updated doc; improved schema for custom insight streams; updated SAT or custom insight streams; removed obsolete optional max_batch_size option from spec |
1.1.11 | 2023-09-21 | 30650 | Fix None issue since start_date is optional |
1.1.10 | 2023-09-15 | 30485 | added 'status' and 'configured_status' fields for campaigns stream schema |
1.1.9 | 2023-08-31 | 29994 | Removed batch processing, updated description in specs, added user-friendly error message, removed start_date from required attributes |
1.1.8 | 2023-09-04 | 29666 | Adding custom field boosted_object_id to a streams schema in campaigns catalog CustomAudiences |
1.1.7 | 2023-08-21 | 29674 | Exclude rule from stream CustomAudiences |
1.1.6 | 2023-08-18 | 29642 | Stop batch requests if only 1 left in a batch |
1.1.5 | 2023-08-18 | 29610 | Automatically reduce batch size |
1.1.4 | 2023-08-08 | 29412 | Add new custom_audience stream |
1.1.3 | 2023-08-08 | 29208 | Add account type validation during check |
1.1.2 | 2023-08-03 | 29042 | Fix broken advancedAuth references for spec |
1.1.1 | 2023-07-26 | 27996 | Remove reference to authSpecification |
1.1.0 | 2023-07-11 | 26345 | Add new action_report_time attribute to AdInsights class |
1.0.1 | 2023-07-07 | 27979 | Added the ability to restore the reduced request record limit after the successful retry, and handle the unknown error (code 99) with the retry strategy |
1.0.0 | 2023-07-05 | 27563 | Migrate to FB SDK version 17 |
0.5.0 | 2023-06-26 | 27728 | License Update: Elv2 |
0.4.3 | 2023-05-12 | 27483 | Reduce replication start date by one more day |
0.4.2 | 2023-06-09 | 27201 | Add complete_oauth_server_output_specification to spec |
0.4.1 | 2023-06-02 | 26941 | Remove authSpecification from spec.json, use advanced_auth instead |
0.4.0 | 2023-05-29 | 26720 | Add Prebuilt Ad Insights reports |
0.3.7 | 2023-05-12 | 26000 | Handle config errors |
0.3.6 | 2023-04-27 | 22999 | Specified date formatting in specification |
0.3.5 | 2023-04-26 | 24994 | Emit stream status messages |
0.3.4 | 2023-04-18 | 22990 | Increase pause interval |
0.3.3 | 2023-04-14 | 25204 | Fix data retention period validation |
0.3.2 | 2023-04-08 | 25003 | Don't fetch thumbnail_data_url if it's None |
0.3.1 | 2023-03-27 | 24600 | Reduce request record limit when retrying second page or further |
0.3.0 | 2023-03-16 | 19141 | Added Level parameter to custom Ads Insights |
0.2.86 | 2023-03-01 | 23625 | Add user friendly fields description in spec and docs. Extend error message for invalid Account ID case. |
0.2.85 | 2023-02-14 | 23003 | Bump facebook_business to 16.0.0 |
0.2.84 | 2023-01-27 | 22003 | Set AvailabilityStrategy for streams explicitly to None |
0.2.83 | 2023-01-13 | 21149 | Videos stream remove filtering |
0.2.82 | 2023-01-09 | 21149 | Fix AdAccount schema |
0.2.81 | 2023-01-05 | 21057 | Remove unsupported fields from request |
0.2.80 | 2022-12-21 | 20736 | Fix update next cursor |
0.2.79 | 2022-12-07 | 20402 | Exclude Not supported fields from request |
0.2.78 | 2022-12-07 | 20165 | Fix fields permission error |
0.2.77 | 2022-12-06 | 20131 | Update next cursor value at read start |
0.2.76 | 2022-12-03 | 20043 | Allows action_breakdowns to be an empty list - bugfix for #20016 |
0.2.75 | 2022-12-03 | 20016 | Allows action_breakdowns to be an empty list |
0.2.74 | 2022-11-25 | 19803 | New default for action_breakdowns , improve "check" command speed |
0.2.73 | 2022-11-21 | 19645 | Check "breakdowns" combinations |
0.2.72 | 2022-11-04 | 18971 | Handle FacebookBadObjectError for empty results on async jobs |
0.2.71 | 2022-10-31 | 18734 | Reduce request record limit on retry |
0.2.70 | 2022-10-26 | 18045 | Upgrade FB SDK to v15.0 |
0.2.69 | 2022-10-17 | 18045 | Remove "pixel" field from the Custom Conversions stream schema |
0.2.68 | 2022-10-12 | 17869 | Remove "format" from optional datetime end_date field |
0.2.67 | 2022-10-04 | 17551 | Add cursor_field for custom_insights stream schema |
0.2.65 | 2022-09-29 | 17371 | Fix stream CustomConversions enable_deleted=False |
0.2.64 | 2022-09-22 | 17304 | Migrate to per-stream state. |
0.2.64 | 2022-09-22 | 17027 | Limit time range with 37 months when creating an insight job from lower edge object. Retry bulk request when getting error code 960 |
0.2.63 | 2022-09-06 | 15724 | Add the Custom Conversion stream |
0.2.62 | 2022-09-01 | 16222 | Remove end_date from config if empty value (re-implement #16096) |
0.2.61 | 2022-08-29 | 16096 | Remove end_date from config if empty value |
0.2.60 | 2022-08-19 | 15788 | Retry FacebookBadObjectError |
0.2.59 | 2022-08-04 | 15327 | Shift date validation from config validation to stream method |
0.2.58 | 2022-07-25 | 15012 | Add DATA_RETENTION_PERIOD validation and fix failed_delivery_checks field schema type issue |
0.2.57 | 2022-07-25 | 14831 | Update Facebook SDK to version 14.0.0 |
0.2.56 | 2022-07-19 | 14831 | Add future start_date and end_date validation |
0.2.55 | 2022-07-18 | 14786 | Check if the authorized user has the "MANAGE" task permission when getting the funding_source_details field in the ad_account stream |
0.2.54 | 2022-06-29 | 14267 | Make MAX_BATCH_SIZE available in config |
0.2.53 | 2022-06-16 | 13623 | Add fields bid_amount bid_strategy bid_constraints to ads_set stream |
0.2.52 | 2022-06-14 | 13749 | Fix the not syncing any data issue |
0.2.51 | 2022-05-30 | 13317 | Change tax_id to string (Canadian has letter in tax_id) |
0.2.50 | 2022-04-27 | 12402 | Add lookback window to insights streams |
0.2.49 | 2022-05-20 | 13047 | Fix duplicating records during insights lookback period |
0.2.48 | 2022-05-19 | 13008 | Update CDK to v0.1.58 avoid crashing on incorrect stream schemas |
0.2.47 | 2022-05-06 | 12685 | Update CDK to v0.1.56 to emit an AirbyeTraceMessage on uncaught exceptions |
0.2.46 | 2022-04-22 | 12171 | Allow configuration of page_size for requests |
0.2.45 | 2022-05-03 | 12390 | Better retry logic for split-up async jobs |
0.2.44 | 2022-04-14 | 11751 | Update API to a directly initialise an AdAccount with the given ID |
0.2.43 | 2022-04-13 | 11801 | Fix user_tos_accepted schema to be an object |
0.2.42 | 2022-04-06 | 11761 | Upgrade Facebook Python SDK to version 13 |
0.2.41 | 2022-03-28 | 11446 | Increase number of attempts for individual jobs |
0.2.40 | 2022-02-28 | 10698 | Improve sleeps time in rate limit handler |
0.2.39 | 2022-03-09 | 10917 | Retry connections when FB API returns error code 2 (temporary oauth error) |
0.2.38 | 2022-03-08 | 10531 | Add time_increment parameter to custom insights |
0.2.37 | 2022-02-28 | 10655 | Add Activities stream |
0.2.36 | 2022-02-24 | 10588 | Fix execute_in_batch for large amount of requests |
0.2.35 | 2022-02-18 | 10348 | Add error code 104 to backoff triggers |
0.2.34 | 2022-02-17 | 10180 | Performance and reliability fixes |
0.2.33 | 2021-12-28 | 10180 | Add AdAccount and Images streams |
0.2.32 | 2022-01-07 | 10138 | Add primary_key for all insights streams. |
0.2.31 | 2021-12-29 | 9138 | Fix videos stream format field incorrect type |
0.2.30 | 2021-12-20 | 8962 | Add asset_feed_spec field to ad creatives stream |
0.2.29 | 2021-12-17 | 8649 | Retrieve ad_creatives image as data encoded |
0.2.28 | 2021-12-13 | 8742 | Fix for schema generation related to "breakdown" fields |
0.2.27 | 2021-11-29 | 8257 | Add fields to Campaign stream |
0.2.26 | 2021-11-19 | 7855 | Add Video stream |
0.2.25 | 2021-11-12 | 7904 | Implement retry logic for async jobs |
0.2.24 | 2021-11-09 | 7744 | Fix fail when async job takes too long |
0.2.23 | 2021-11-08 | 7734 | Resolve $ref field for discover schema |
0.2.22 | 2021-11-05 | 7605 | Add job retry logics to AdsInsights stream |
0.2.21 | 2021-10-05 | 4864 | Update insights streams with custom entries for fields, breakdowns and action_breakdowns |
0.2.20 | 2021-10-04 | 6719 | Update version of facebook_business package to 12.0 |
0.2.19 | 2021-09-30 | 6438 | Annotate Oauth2 flow initialization parameters in connector specification |
0.2.18 | 2021-09-28 | 6499 | Fix field values converting fail |
0.2.17 | 2021-09-14 | 4978 | Convert values' types according to schema types |
0.2.16 | 2021-09-14 | 6060 | Fix schema for ads_insights stream |
0.2.15 | 2021-09-14 | 5958 | Fix url parsing and add report that exposes conversions |
0.2.14 | 2021-07-19 | 4820 | Improve the rate limit management |
0.2.12 | 2021-06-20 | 3743 | Refactor connector to use CDK: - Improve error handling. - Improve async job performance (insights). - Add new configuration parameter insights_days_per_job . - Rename stream adsets to ad_sets . - Refactor schema logic for insights, allowing to configure any possible insight stream. |
0.2.10 | 2021-06-16 | 3973 | Update version of facebook_business to 11.0 |
0.2.9 | 2021-06-10 | 3996 | Add AIRBYTE_ENTRYPOINT for Kubernetes support |
0.2.8 | 2021-06-09 | 3973 | Add 80000 as a rate-limiting error code |
0.2.7 | 2021-06-03 | 3646 | Add missing fields to AdInsights streams |
0.2.6 | 2021-05-25 | 3525 | Fix handling call rate limit |
0.2.5 | 2021-05-20 | 3396 | Allow configuring insights lookback window |
0.2.4 | 2021-05-13 | 3395 | Fix an issue that caused losing Insights data from the past 28 days while incremental sync |
0.2.3 | 2021-04-28 | 3116 | Wait longer (5 min) for async jobs to start |
0.2.2 | 2021-04-03 | 2726 | Fix base connector versioning |
0.2.1 | 2021-03-12 | 2391 | Support FB Marketing API v10 |
0.2.0 | 2021-03-09 | 2238 | Protocol allows future/unknown properties |
0.1.4 | 2021-02-24 | 1902 | Add include_deleted option in params |
0.1.3 | 2021-02-15 | 1990 | Support Insights stream via async queries |
0.1.2 | 2021-01-22 | 1699 | Add incremental support |
0.1.1 | 2021-01-15 | 1552 | Release Native Facebook Marketing Connector |