AWS Database Migration Service (AWS DMS)

The AWS Database Migration Service (DMS) offers a streamlined solution for transitioning on-premise databases to the cloud, leveraging AWS advantages like enhanced performance and cost transparency. While DMS simplifies the migration process, it necessitates preliminary setup and a deep understanding of its components. By migrating to AWS, businesses can expedite feature delivery by up to threefold, curtail maintenance costs by approximately 31%, and elevate IT staff productivity by 62%. In this guide, we'll delve into the intricacies of AWS DMS, highlighting its merits and key components. Additionally, we'll outline a step-by-step approach for migrating an On-Premises Oracle Database to Amazon Aurora MySQL, ensuring a seamless transition.

How Does It Work?

The regulated and automated migration service provided by AWS DMS enables the transfer of data between databases. The method begins with integrating DMS with the endpoints (target and source). To utilize AWS DMS, at least one of your endpoints must be hosted on an AWS service.

• Consider AWS DMS to be an AWS cloud server that houses replication software. You must specify a source connection and a destination connection to instruct AWS DMS where to extract and transfer the data.
• You may now plan a server-side process to move your data. If the required tables and primary keys don't already exist at the destination, AWS DMS will create them.
• It is also possible to build the target tables from scratch. It is possible to generate the target tables, indexes, views, triggers, and other objects using the AWS SCT (AWS Schema Conversion Tool).

• Making a link to your data source, retrieving your data, and organizing it for the database destination are the first steps in the AWS Database Migration process.
• The data is subsequently imported into the target database. Data from the source is sent to the target during a full load migration. Any table updates are kept as cached modifications on the replicating server for times when there is a high volume of traffic.
• AWS DMS begins applying the cached modifications to the database as soon as the whole load is complete, keeping the source and destination databases in sync.

Components Used in Data Migration with AWS DMS

AWS DMS consists of three components. Let's discuss them in detail.

• Replication instance
• Source & Target Endpoints
• Replication tasks

Replication Instance

• A managed instance of the Amazon Elastic Compute Cloud (EC2) that houses one or more replication jobs is known as a replication instance.

Replication instances are of various types:

• T2/T3: For configuring, designing, and testing database migration methods, these were made. Additionally, these instances can be used for ongoing migration processes.
• C4: These instances are suited for workloads that require a lot of computation and are performance-optimized. Heterogeneous migrations are suitable in these situations.
• R4/R5: These memory-optimized instances ought to be utilised for high-throughput transactions and ongoing migrations.

• The graphic above shows a replication server doing a number of connected replication jobs.

Source and Target Endpoints

• Endpoints are used by the AWS DMS to transport data and connect source and target databases.
• Based on the database, the kind of endpoint will vary, but they all need the same data, including the type of engine, credentials, encryption techniques, port number, hostname, and endpoint type.

Replication Tasks

• The replication job is used to move data from the source end to the destination endpoint, allowing you to specify which tables and schemas are migrated and when.
• The replication job creation phase must be accomplished before beginning a migration.
• When creating a replication job, all the necessary credentials, and the replica instance to use must all be provided.

Ongoing Replication, or Change Data Capture (CDC)

• The replication job in the CDC procedure is made to stream updates from the source to the target while employing in-memory buffers to store data while it is being transferred.
• The replication process will leak any unfinished modifications to the disk's Change Cache if the in-memory buffers run out of space.
• For instance, if AWS DMS is recording changes from the source more quickly than they can be implemented on the target, this can happen. You will observe that the task's target latency is greater than its source delay in this instance.

Schema and Code Migration

• The conversion of codes or schemas is not done by AWS DMS. If your source and target databases use the same database engine, you can migrate your schema using programs like Oracle SQL Developer, MySQL Workbench, and pgAdmin III.
• AWS SCT can be used to change an existing schema to a different database engine.

Types of AWS Data Migrations

Primarily there are two types of AWS Data Migrations

Homogeneous Database Migration

When the source and target databases are identical or consistent with one another, as in the cases of MySQL to Amazon RDS for MySQL, MySQL to Amazon Aurora, Oracle to Amazon RDS for Oracle, or Microsoft SQL Server to Amazon RDS for SQL Server, homogeneous database migration takes place. Given that the destination and source databases schema structures and data types are same, this is a one-step process.

Heterogeneous Database Migration

• Heterogeneous database migration occurs when the source and target database engines are different, for example when switching from Microsoft SQL Server to MySQL, Oracle to PostgreSQL, or Oracle to Amazon Aurora.
• Since the schema structures and data types between the source and target databases differ, a schema and code transformation is required prior to transfer. As a result, migration is a two-step process.

AWS Data Migration Type Options

AWS DMS offers three different types of migration.

Full Load

Full Load with AWS DMS migrates all of the data currently in your database; it does not copy data updates. This is a great choice if you simply want to migrate once and do not need to record continuing changes.

Full Load + CDC

Another AWS DMS choice is full load + Change Data Capture(CDC), which will migrate all of your data firstly and replicate any later changes at the source as well. While the operation is being performed, your database will be monitored. When you have really large databases and don't want to pause tasks, this is very beneficial.

CDC Only

Only using AWS DMS, CDC will only duplicate database updates, not the initial complete load of data. This approach is appropriate if you are transferring your database through another technique but still want to keep up with changes occurring at the source.

Features of AWS DMS

1. AWS DMS Fleet Advisor

AWS DMS Fleet Advisor is a free, fully-managed service of AWS DMS. You can swiftly and effectively deploy analytics and database deployments to the cloud at scale by automating migration processes. Fleet Advisor helps you create a distinctive migration plan by estimating how challenging it will be to move your origin databases to the target services on AWS.

To know more please look into the link mentioned here https://aws.amazon.com/dms/fleet-advisor/

Migration Tasks That AWS DMS Performs

An AWS DMS task is used to carry out the procedure. You define the tables , structures, and underlying data to use for your migration in addition to any special processing, such as logging requirements, control table data, and error handling.

Three main stages can make up a task:

• Migration of existing data (Full load)
• The application of cached changes
• Replication (Change Data Capture)

Getting Started with AWS Database Migration Service

To perform a database migration, take the following steps:

• Follow the instructions in Setting up for AWS Database Migration Service to set up your AWS account.
• To fill your source database and test replication, make your test databases and an Amazon EC2 client. Additionally, to house your lesson resources, construct a virtual private cloud (VPC) based on the Amazon Virtual Private Cloud (Amazon VPC) service. Follow the instructions in Prerequisites for AWS Database Migration Service to build these resources.
• Utilize a sample database creation script to populate your source database.
• To transfer the schema from the source database to the destination database, use the AWS Schema Conversion Tool (AWS SCT). Follow the instructions in Migrate schema to convert the schema.
• Create a replication instance and run all the migration operations on it. Follow the Replication instructions to complete this and the other tasks.
• Give the endpoints for the source and target databases. See Generating source and target endpoints for more on creating endpoints.
• Create a task to specify the replication processes and tables you want to use, then begin replication. See Generating a task for details on creating database migration tasks.
• By executing queries on the target database, confirm that replication is operational.

For more information on getting started please refer to this link https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.html.

AWS DMS Supported Databases

Source Database Engines

• Oracle Database and Data Warehouse
• Microsoft Azure SQL Database and Server
• Teradata
• IBM Netezza
• Greenplum
• HPE Vertica
• MySQL
• PostgreSQL
• IBM DB2 LUW
• Apache Cassandra
• SAP ASE
• Amazon Redshift
• Azure Synapse Analytics
• Snowflake

Target Database Engines

• Amazon Aurora MySQL
• Amazon Aurora PostgreSQL
• MariaDB
• MySQL
• PostgreSQL
• Amazon Redshift
• Aurora MySQL
• Aurora PostgreSQL
• Microsoft SQL Server
• Amazon DynamoDB

AWS DMS Use Cases

Move to Managed Databases

Utilize a streamlined migration procedure to move from old or on-premises databases to managed cloud services, freeing up development time.

Eliminate Licensing Fees to Speed Business Growth.

Utilize modern, purpose-built databases to develop and build more quickly and affordably for any use case.

Replicate Backup Files

To reduce downtime and data loss, create redundant copies of business-critical databases and data repositories.

Data Mart Integration

Create data lakes and handle change data from your data repositories in real time.

AWS DMS Pricing

You solely need to pay for your replication instances and any extra log storage. Each database migration instance has ample storage for swap space, replication logs, and data caching for the most of replications, and inbound data transfer is not charged.

On-Demand Instance Pricing

The AWS Database Migration Service can now support the T2, T3, C4, C5, R4, and R5 instance types. T2/T3 instances are low-cost standard instances with a burst capacity capable of surpassing the CPU performance threshold. The instances C4, R4, and R5, are designed for optimal CPU performance. Using C4 or R4/R5 instances will help you quickly move an extensive database.

Let’s take a few examples of the T2/T3 series –

T3 CPU Credits

CPU Credits are charged at $0.075 per vCPU-Hour. For all T3 instance sizes across all geographies, the CPU Credit price is the same.

Capacity Pricing

Each C4 or R4 and R5 instance type provides 100GB of GP2 network-attached storage for swap space, replication logs, and data cache. GP2 network-attached storage adds up 50GB for each T2 and T3 instance type.

Data Transfer Pricing

It costs nothing at all to transfer data into the AWS Database Migration Service. On the same Availability Zone, data transfers between AWS DMS and databases in Amazon RDS and Amazon EC2 instances are likewise free. Standard AWS data transfer fees apply when moving your source database to a destination database that is in a different Availability Zone, Region, or outside of AWS.

AWS DMS Fleet Advisor

The Amazon DMS's AWS DMS Fleet Advisor is a completely free feature (AWS DMS). By automating inventories and offering personalised migration guidance, DMS Fleet Advisor streamlines database and analytics migration planning.

Benefits of AWS DMS

• Because AWS DMS is serverless and can deploy, maintain, and oversee all the hardware and software needed for migration automatically, you can do away with tedious chores like performance evaluation, hardware and software sourcing, system installation and administration, and framework testing and debugging. Technically, depending on how AWS DMS is configured, your transfer may begin shortly.

• Minimal downtime -- DMS continuously updates your data source throughout the migration process while ensuring its functionality. As a result, you are not required to stop to switch between databases.

• Reliability – The self-healing Database Migration Service will resume immediately away in the occurrence of an interruption. DMS allows you to build up a Multi-AZ (availability zone) replica for disaster response.

• Supports Most Notable Databases – AWS Database Migration Service can help you migrate your data from the most prominent public and corporate databases.

• Low cost – To migrate to DocumentDB, Redshift, Aurora, or DynamoDB, use the free Data Migration Service. Depending on the amount of log retention and the computational load, you must purchase additional databases.

Limitations of AWS DMS

Problems with AWS DMS Schema Migration and Conversion

Schema or code transformation is frequently not performed by AWS DMS. AWS DMS attempts to establish a target schema at the destination when transferring data as part of a homogenous migration. With Oracle databases, for example, this isn't always feasible.

You might need to utilize third-party tools to produce schema. AWS DMS cannot convert schemas in the event of heterogeneous migrations; the AWS Schema Conversion Tool must be used instead (SCT). Not all formats are supported, but doing so enables you to change your source schema to one appropriate for your intended use. Conversions must be performed manually if the tool does not support your format.

Coding Specifications for Incremental Data Collection

The AWS DMS requires some programming, which might be labor- and time-intensive, to capture incremental loads. For instance, S3 changes need to be integrated using a particular code.

Performance

AWS DMS by default supports loading 8 tables concurrently. This might somewhat improve speed when using a very large replication server. The replication, however, could take longer if the source generates data at a high throughput.