AWS Appflow

Introduction to AWS Appflow

Amazon AppFlow is a fully managed integration tool that allows you to securely exchange data between SaaS(Software as a service) apps like Salesforce and Amazon Web Services (AWS) services like Amazon Simple Storage Service (Amazon S3) and Amazon Redshift. For example, you can feed Salesforce contact records into Amazon Redshift or Zendesk support tickets into an Amazon S3 bucket. The graphic below demonstrates how it works.

What Does AWS Appflow Let You Do?

You can perform the following using Amazon AppFlow: Get started right away – Create data flow in minutes without writing any code to transmit data between a source and a destination.

• Sync your data – Run flows on-demand or on a schedule to synchronize data across your SaaS apps and AWS services.
• Bring your data together – Combine data from numerous sources to improve the effectiveness of your analytics tools and save money.
• Keep track of your data – By using Amazon AppFlow flow management tools, track where and when data has migrated.
• Keep your information safe – Safety is paramount. It encrypts your data both in transit and at rest.
• Secure data transfer – AppFlow interfaces with AWS PrivateLink to offer private data transfer via AWS infrastructure rather than public data transfer over the internet.
• Create custom connectors – You can create custom connectors with the AWS AppFlow Custom Connector SDKs(Software Development Kit) for data sources that aren't yet connected with the service. Using connectors, you may move data between private APIs, on-premise systems, other cloud services, and AWS.

Features

Ease of Use

Point and Click User Interface

No coding is necessary when using AWS AppFlow to set up data flows. You may choose your data sources and destinations, set up optional transformations and validations, and run your flow without relying on technical teams, thanks to a point-and-click user interface.

Flexible Data Flow Triggers You can set up a regular schedule to keep your data in sync with AWS AppFlow, or you can conduct data flows in reaction to business events like the creation of a sales opportunity, a change in the status of a support ticket, or the submission of a registration form.

Native SaaS Integrations

Salesforce, Marketo, Slack, and other Software-as-a-Service (SaaS) applications are just a few of the SaaS applications natively integrated with AWS AppFlow, and more integrations are planned. With only a few clicks and AppFlow, you can quickly move data from any compatible SaaS application.

Easy to Use Field Mapping You can bulk map source and destination fields using the AWS AppFlow interface or map each area individually. You can also upload a CSV file to map numerous fields, for data flows with many fields.

Cost Savings

Pay as You Go

Compared to developing connectors internally or utilizing alternative application integration services, AWS AppFlow offers a substantial cost advantage. Customers only pay for the number of flows they run and the amount of data they handle with AppFlow; there are no setup fees or license costs associated with using the service.

Scalable

High Scale Data Transfer

You can quickly move millions of Salesforce records, Marketo leads, or Zendesk tickets using Amazon AppFlow because it can process up to 100 GB of data per flow.

Enterprise Grade Data Transformations

There is no need for additional steps when using AWS AppFlow to execute data transformations, including mapping, merging, masking, filtering, and validation. For instance, you can merge first and last names, verify that data is in the appropriate numeric format, or hide credit card information.

Secure & Reliable

Default Data Privacy Through Private Link By preventing data from being exposed to the public Internet, AWS PrivateLink streamlines the security of data shared with cloud-based apps. AppFlow automatically builds and configures private endpoints for SaaS applications that have PrivateLink enabled, ensuring that your data is kept private by default.

Custom Encryption Keys You can use AWS keys or your custom keys to encrypt data because all data traveling via AWS AppFlow is encrypted both in transit and at rest.

Iam Policy Enforcement

With AWS AppFlow, you can enforce fine-grained permissions and maintain access consistency throughout your business without writing new policies by using your existing AWS identity and access management (IAM) policies. Application administrators may securely build and manage data flows without relying on technical teams with the help of carefully enforced IAM policies.

Built-in Reliability Amazon AppFlow is built with a highly available architecture to prevent single points of failure. Amazon AppFlow takes advantage of AWS scaling, monitoring, auditing, and billing features so there’s no need to configure these yourself.

AWS Appflow Use Cases

Get 360-Degree Visibility of Customer Journey

Using AWS AppFlow, you can combine data from sales, customer service, and marketing to gain a complete picture of the customer journey. By syncing revenue data from Salesforce with marketing data from Marketo, you may, for instance, assess the revenue effect of various marketing channels.

Data Enrichment

AWS AppFlow can be used to improve data for various SaaS services. For instance, import Salesforce data into Amazon Sagemaker's unique machine learning model to produce priority tags for situations and send those tags back into Salesforce to route suitable sales help.

Create Event-Based Workflows

Instead of doing so manually, you can utilize AWS AppFlow to start processes automatically in one application depending on data from another. Make contact records in Salesforce, for instance, using fresh Marketo leads.

Store or Sync Salesforce Data

You may utilize AWS AppFlow to move Salesforce opportunity record data to an Amazon Redshift table for real-time dashboard updates. Even better, you can use AppFlow to synchronize data between two Salesforce org, such as your finance and marketing org, for budgeting purposes.

Automate Data Backups

Data from SaaS apps can be automatically backed up in Amazon S3 using AppFlow. For further root cause analysis, set up a flow to send high-severity Zendesk tickets to Amazon S3.

Regularly Examine Slack Events

Sending events and discussion data to a business intelligence solution will enable you to set up a routine analysis of instant messenger chats. For instance, you could use AWS AppFlow to build a data flow that routes Slack Channel event and conversation data to an Amazon Redshift or Snowflake data warehouse for later analysis.

Data Security with AWS Appflow

The data protection in AWS AppFlow complies with the AWS shared responsibility model. According to this paradigm, AWS is in charge of safeguarding the global infrastructure that powers the whole AWS Cloud. Your content hosted on this infrastructure, its management, and security must be your responsibility.

Encryption at Rest

A connection is made when you set up a SaaS application as a source or destination. This information is necessary for connecting to SaaS apps along with user names, passwords, and authentication tokens. Your connection data is securely stored by AWS AppFlow after being encrypted using Customer Master Keys (CMK) from AWS Key Management Service (AWS KMS) and kept in AWS Secrets Manager.

All of a connection's metadata is permanently removed when you delete it. You can use either an AWS-managed CMK or a customer-managed CMK when utilizing Amazon S3 as a destination to encrypt the data in the S3 bucket using Amazon S3 SSE-KMS.

Encryption in Transit

When configuring a flow, you can select either an AWS-managed CMK or a customer-managed CMK. Amazon AppFlow uses this key to encrypt data and temporarily store it in an intermediary S3 bucket before starting a flow. Utilizing a bucket lifecycle policy, this intermediate bucket is removed after seven days.

Utilizing Transport Layer Security (TLS) 1.2, Amazon AppFlow protects all data in transit. You can establish a connection without using the open internet with select SaaS applications that are supported sources or destinations.

Key Management

When Amazon S3 is a destination, AWS AppFlow offers both AWS-managed and customer-controlled CMKs for encrypting connection data and data stored there. Since a customer-controlled CMK gives you complete control over your protected data, we advise using one. A resource policy that gives Amazon AppFlow access to the customer-managed CMK is attached when you select it.

Connection Credentials

Amazon AppFlow stores the encrypted credentials that are used to connect to flow source and destination applications in your AWS Secrets Manager account. These credentials include OAuth tokens, Application and API keys, and passwords. To create a new connection, grant the following permissions to any custom IAM policies.

The AmazonAppFlowFullAccess policy includes these permissions.

Getting Started

This section introduces Amazon AppFlow and describes how to create a flow using the AWS console. The diagram above shows how to move and enrich data from a data source to a data destination in your flow using Amazon AppFlow.

Prerequisites Required

Before you begin using Amazon AppFlow, you must first complete the following criteria:

• AWS account setup - If you don't already have an AWS account, you'll need to create one.
• Setup of SaaS applications - Make sure you have all of the necessary information regarding the source and destination applications and that they meet all of the appropriate configuration criteria.
• Identity and access management - If you use AWS as an IAM user, your administrator must provide you with the permissions necessary to construct and run flows.
• AWS CloudFormation OAuth (Optional) - Suppose you wish to utilize AWS CloudFormation to construct a connection profile for connectors that use OAuth (such as Salesforce, Slack, Zendesk, and Google Analytics). In that case, you'll need to retrieve the access and refresh tokens first. You can create your OAuth user interface or obtain them from another source. Alternatively, you can construct the connector profile in the Amazon AppFlow dashboard and then utilize it in the AWS CloudFormation template for flow formation.
• Data encryption (optional) - During transit and at rest, Amazon AppFlow encrypts your data and connection details. You provide an AWS Key Management Service CMK for encryption when configuring a flow. You can use the AWS-managed CMK titled AWSDefaultEncryptionKey that Amazon AppFlow creates by default or construct your own customer-managed CMK.

Creating Flow using AWS Console

The AWS console user interface, AWS CLI commands, APIs, or specifying CloudFormation resources are all options for getting started with generating your first flow. The console allows you to enter basic flow information and connect to the related SaaS application.

To Use the Console to Design a Flow The procedures to create and configure a flow using the Amazon AppFlow console user interface are outlined in the following process.

• Go to Amazon AppFlow Console to access Amazon AppFlow.
• Select Create Flow.
• Enter a name and description for the flow in the Flow details section. An acceptable flow name contains alphanumeric characters and special characters like !@#.-_.
• (Optional) Choose Data encryption, Customize encryption settings, and then select an existing CMK (Customer Master Key) or build a new one instead of using the default AWS managed CMK.
• (Optional) Go to Tags, Add tag, and input the key name and value to create a tag.
• Select Next.

To Set up The Flow

• Select the source and enter the relevant information for Source Details. Provide connection information and select objects or events, for example. For more information, go to the Supported source and destination applications page and look up your source application to find application-specific connection instructions.
• Keep in mind that to successfully configure a connection for a flow, the user or role you use to construct the flow must have permission to utilize the UseConnectorProfile permission-only action for the flow's connection (connectorprofile). The AmazonAppFlowFullAccess managed policy includes this permission. If you're using a custom policy, you'll need to add the connector profile resource and permission to the policy.
• Select the destination and give the relevant information about the place in the Destination Details section. For more information, go to the Supported source and destination applications page and look up your destination application. You'll find application-specific connection instructions there.
• Choose how to start the flow in the Flow trigger. The flow trigger possibilities are as follows:
  • Run-on Demand - Run the flow manually or on-demand.
  • Run-on Event - Run the flow in response to the provided change event.
    • This option is only accessible for SaaS apps that support change events. When you select the source, you must also select the event.
  • Run-on Schedule - Run the flow according to the parameters and transfer the data.
    • For schedule-triggered flows, you can choose between full or gradual transfer.
    • When you choose full transfer, Amazon AppFlow transmits a snapshot of all records from the source to the destination at the time of the flow run.
    • When you choose incremental transfer, Amazon AppFlow only sends the records that have changed or been added since the last successful flow run. You may also configure how Amazon AppFlow recognizes new or updated entries by selecting a timestamp field. Choose this option if you have a Created Date timestamp column to tell Amazon AppFlow to only transfer freshly created entries (not updated data) since the last successful flow execution. At the time of the first flow run, the first flow in a schedule-triggered flow will pull 30 days of historical records.
    • The frequency of the scheduling is determined by the frequency supported by the source application.
• Select Next.

Mapping Data Fields

• Select a mapping method for the fields and complete the field mapping. The field mapping possibilities are as follows:
  • Manually map fields - Specify field mapping using the Amazon AppFlow user interface. Choose Source field name, Bulk actions, and Map all fields directly to map all fields. Select one or more fields from the Source field name, and Source fields, and then directly choose Map fields.
  • Upload a .csv file containing mapped fields - To specify field mappings, use a comma-separated values (CSV) file. The source field name appears first in the CSV file, followed by a comma, and then the destination field name.
• (Optional) To add a formula that concatenates fields, pick Add formula after selecting two fields from Mapped fields.
• (Optional) Select one or more fields from Mapped fields and then Modify values to mask or truncate field values.
• (Optional) Add validations to verify if a field has bad data. Select the criterion that signals faulty data for each field and the action that Amazon AppFlow should do when a field in a record is bad.
• Select Next.

Adding Filters

To select which records to transfer, apply a filter. You can filter data fields in Amazon AppFlow by using multiple filters and adding criteria to a filter.

• Select Add filter, select the field name, select a condition, and then specify the criteria to add a filter.
• (Optional) Select Add criteria to add more criteria to your filter. You can add up to 10 criteria per filter, depending on the field and condition.
• (Optional) Select Add filter again to add another filter. You can choose which data fields to use in your flow using up to ten filters. Amazon AppFlow will apply each filter in the order you set it, and only records that fulfill all filter criteria will be transferred.
• Choose Remove, next to the filter, to remove it.
• Choose Next once you've completed adding filters.
• Examine the data in relation to your flow. Choose Edit to alter the information for a step. When you're done, select Create flow.

Pricing

Significant cost reductions are possible when comparing AWS AppFlow to in-house connector development or other application integration providers. Customers only pay for the number of flows they run and the amount of data they handle; there are no setup fees or ongoing costs associated with using AWS AppFlow.

AppFlow cost would depend on 2 factors:

• Number of flows run in a month – AppFlow charges $0.001/flow
• Data processing fee for each flow – This is calculated based on the volume of data processed each month. This charge varies with the region of your S3/Redshift destinations.

In addition to the above, AWS would also levy standard requests and storage charges to read and write from AWS services such as Amazon S3.

Other AWS Services that Can Be Used with Appflow

Amazon S3

By providing bi-directional data transfer for workflow activation, downstream analysis, and database syncing, Amazon AppFlow enables you to fully utilize the storage capabilities of Amazon S3 in conjunction with other SaaS apps like Zendesk, Salesforce, and ServiceNow.

Amazon EventBridge

A serverless event bus called Amazon EventBridge makes it simple to link apps utilizing information from your applications, integrated SaaS applications, and AWS services. EventBridge transmits real-time data to AWS targets from event sources, including ZenDesk, DataDog, and AWS Lambda. Users of Amazon EventBridge can create integrations with all SaaS programs that are a part of AppFlow, including Salesforce, and receive events from those programs.

Amazon Redshift

Data may be seamlessly transferred to Amazon Redshift's data-warehousing service from various SaaS apps utilizing Amazon AppFlow. For example, you can establish data flows from Slack Channel to Amazon Redshift to analyze event data. Alternatively, you may move record data from Salesforce to Amazon Redshift to update real-time dashboards.

Amazon Connect Customer Profiles

By utilizing detailed customer profiles, Amazon Connect Customer Profiles enables enterprises to interact with customers to provide personalized customer service and support. Utilize Amazon AppFlow to increase customer awareness by integrating CRM, customer ticketing, and customer communications applications.

Amazon Honeycode

Teams can create effective mobile and web applications without writing any code using Amazon Honeycode. Teams can quickly link their Honeycode apps with other SaaS and AWS services using Amazon AppFlow. For instance, sales teams can create an application that syncs data between Honeycode and Salesforce.

Amazon Lookout for Metrics

A machine-learning service called Amazon Lookout for Metrics identifies outliers in financial and operational metrics, isolates their underlying causes, and enables users to act swiftly without the need for special machine-learning knowledge. Utilizing Amazon Lookout for Metrics, you can use Amazon AppFlow to obtain data from various SaaS and Cloud apps, like Salesforce, Google Analytics, ServiceNow, etc., and turn that data into actionable insights.