The most forward-looking companies are adopting cloud solutions and choose cloud data migration for Data Science and Business Intelligence since that offers a tangible advantage over the competition. What’s more, the risks associated with data migration to cloud are now reduced to the minimum.
Depending on the specific business needs, companies opt for moving to private, public or hybrid clouds. The majority of companies choose public clouds because they offer such benefits as pay-as-you-go scalability, cost-effectiveness, and reliability. However, if a company is concerned with moving sensitive data to the cloud, a hybrid cloud could be a perfect solution, in which the most sensitive data will be hosted on-premises to avoid potential security risks, and the rest of the data will be moved to a public cloud.
Source: The 2017 State of the Cloud Survey
There are several vendors of public clouds such as AWS, Azure, Google Cloud, IBM etc. According to the The 2017 State of the Cloud Survey, AWS continues to be the leading cloud service provider (57 percent of respondents currently run applications in AWS).
Source: The 2017 State of the Cloud Survey
Key reasons to use public clouds for cloud data migration:
- Optimization of resource usage and scalability. You can increase/decrease resources (RAM, CPU, storage etc.) any time a need arises. This is not the case with traditional shared, VPS or dedicated hostings. Thus it is beneficial for businesses that need to scale up for peak seasons or time of day. Scalability means also more flexibility and less limitations regarding the amount of data to be stored and processed at one time.
- Cost-efficiency since you don’t need to pay for maintenance of on-premises infrastructure. Cloud computing services are paid on-demand and entail lower infrastructure and maintenance costs. If you are a company working with legacy systems, it can save you considerable costs.
- You can use different types of databases. The cloud data warehouse migration is a good way to use new databases and new data models such as SQL and NoSQL.
- Most public cloud providers guarantee automatic security updates and keep traffic within tolerable limit at peak times.
- Automatic backup and logging of the key metrics. In case of a disaster recovery, the backups will enable you to resume the working processes, and the logs will provide you with the necessary details regarding what caused the problem.
Cloud data migration with Amazon Web Services for Data Science and BI. 3 best practices:
- Lift-and-shift cloud migration model. It is implemented by DevOps migrating the whole instances without changing the code. Your source is the database you want to move data from and the target is the database you are moving your data to. The replication instance processes the data migration tasks and requests access to your source and target databases inside your Amazon Virtual Private Cloud. That is the case if you need to migrate quickly (for example, in case of a data center lease expiry). It takes minimal effort and time to migrate. However, it may cost more to operate in the cloud and you won’t be able to take advantage of the native features of the cloud platform.
- Partial rewriting in the cloud. In this case only parts of the application are modified. It entails faster migration and deployment than complete rewriting. It means taking advantage of some of the native features of the cloud, and it may be costlier to operate it in the cloud. N-iX experts have performed partial data warehouse rewriting for one of our clients, Lebara, a global telecom company.
- Complete rewriting in the cloud. Completely rewritten databases typically offer higher performance and can be optimized to operate at lower costs. Writing a cloud-native data warehouse means using the best of AWS data ingestion, transformation, load-balancing, scaling, and storage services such as Kinesis, S3, RedShift, Lambda & API gateway etc. We have performed complete database rewriting for one of our clients, Gogo, a global leader in in-flight connectivity. Our experts rewrote the customer’s warehouse database from SQL/SQL Server Integration Services to Spark, Python, and Scala. That enhanced performance and scalability.
Key challenges of cloud data migration:
- It entails a lot of first-rate development expertise. If you rewrite the system with mistakes, it will incur even more costs than maintaining the infrastructure of the on-premises system. So the key challenge is finding the right experts and establishing a team of excellence.
- The security concern. Healthcare companies used to have concerns regarding keeping their data in the cloud. However, now, cloud hosting is compliant with HiPAA regulations. Yet, if you have concerns regarding sensitive data, a hybrid model could be a perfect solution, in which the most sensitive data will be hosted on-premises or in a public cloud, and the rest of the data will be migrated to a public AWS cloud.
- Public cloud services can incur unexpected costs. Here are 10 tactics to manage them.
Source: RightScale 2017 State of the Cloud Report
Expertise you need for cloud data migration:
- For data warehouse migration you need to have specialists with expertise in Big Data Stack, Python, Scala, SQL, Apache Spark; understanding of the business model of the legacy system; knowledge of many open-source tools and technologies.
- Expert business analysts with good understanding of the business model of your legacy system.
- DevOps specialists with solid experience in migrating data warehouse to the cloud platform and good knowledge of using and optimizing cloud services.
Wrap-up
Many companies opt for cloud data migration with AWS for Data Science and BI or implement hybrid solutions since that offers such benefits as scalability, cost-efficiency and reliability. Depending on your needs you may opt for lift-and-shift practice, partial rewriting or complete rewriting. The key challenge is to to find the necessary expertise and specialists with the knowledge in Big Data stack, Python, Scala, SQL, understanding of the business model of the legacy system, and open-source technologies. N-iX data scientists and BI specialists have solid experience in data warehouse migration to cloud. Consult us on any pertinent question!
