Skip to content
What is Amazon Redshift?
Redshift, the AWS petabyte data warehouse solution, is designed to offer fast query performance with the use of columnar storage technology and is available to use over a wide range of SQL clients. There are two Redshift snapshot types: automated and manual, and both are stored in Amazon S3. In this article, let’s dig deeper on the need to use and automate Redshift backups using manual snapshots.
What Are Amazon Redshift Snapshots and Backups?
Amazon Redshift snapshots are backups of your Redshift clusters. They capture the state of the cluster at a specified point in time, including the data and configuration settings. Snapshots are crucial for data protection, enabling you to restore your Redshift cluster to a previous state in case of data corruption, user errors, or system failures.
There are two main types of Redshift snapshots:
- Automated snapshots: These are taken automatically by Amazon Redshift at regular intervals. By default, Amazon Redshift retains these snapshots for a short period (usually one to five days), after which they are deleted. Automated snapshots are primarily used for routine cluster recovery within a defined retention period.
- Manual snapshots: Unlike automated snapshots, manual snapshots are created by users at any time and can be retained for an indefinite period. This provides flexibility in keeping backups of specified points in time, particularly when performing important database changes or updates.
Both snapshot types are stored in Amazon S3, offering durability and scalability. You can restore your Redshift cluster from these snapshots to a new cluster, or you can restore individual tables or databases from the snapshot without affecting other parts of your cluster.
This is part of a series of articles about AWS Backup.
Related post: AWS backup vs snapshot
3 Reasons to Automate Your Manual Redshift Snapshots
1. Manual snapshots are crucial for longer retention
Redshift’s automated backups are held for a maximum retention period of up to 35 days, then are automatically deleted. Manual snapshots, on the other hand, can be saved for as long as needed, are not automatically deleted and can be taken at any desired point in time. Manual snapshots can be crucial when it comes to complying with strict regulation rules that generally require data to be saved for a number of years. Additionally, manual snapshots can be particularly useful if you load a large amount of data into your Redshift cluster, for example, and want to immediately back it up.
Furthermore, AWS limits the amount of manual snapshots you can take to 20 snapshots per account. If you have multiple databases, you might want to raise your snapshot limit. Contact AWS or carefully refine your backup policy to take snapshots at larger intervals.
2. Surviving an Altered or Terminated Cluster
According to this AWS re:post discussion, automated Redshift backups are taken approx. every eight hours. If a database is altered, there is a good chance you won’t be able to restore it to its most recent changes. In which case, you should make sure that a manual snapshot has been taken. Redshift deletes automated snapshots when you delete the cluster. This means that your Redshift database cannot be restored because the snapshot is lost. At the moment of deletion, you are prompted to capture a manual snapshot, although the significance of this can sometimes be lost and this stage bypassed, meaning there will be no database backup.
Manual Redshift snapshots on the other hand, are stored separately, and if used, are the only way to save a deleted cluster. In both cases of automating your manual snapshots, you will be able to set your backup policy including high-frequency snapshots. Aside from controlling the policy, automating manual snapshots creates seamless and transparent backups, which increase confidence in Redshift operations, and allows you to perform restorations at any point in time.
3. Cross-Region Backup
In the event of an outage, AWS disaster recovery plays a key role in making sure backups are safe and secure in case a region goes down.
Your retention periods set for automated snapshots in AWS destination regions might be different than those within the cluster’s source region. Each AWS region has its own specific retention period for automated snapshots. In addition, cross-region snapshots are limited to a retention period of seven days. If you wish to keep and maintain a comprehensive backup of your Redshift cluster (specifically for mission critical production environments) in another region for high availability purposes, manual snapshots are the better option.
N2W Backup & Disaster Recovery offers a fail-safe solution that automatically configures the N2W policy to schedule an automated backup however frequently you choose. It automatically copies the snapshot into a separate region as well as performs additional operations such as cross-account backup, copy to S3, and resource control which allows you to schedule the start/stop of your resources for cost savings.
- Combine S3 storage classes for snapshot archiving: For long-term manual snapshots, consider using storage classes like S3 Glacier or Deep Archive. This can significantly reduce costs while maintaining the ability to restore.
- Optimize Redshift clusters before snapshots: Schedule VACUUM and ANALYZE operations before taking manual snapshots to minimize unused storage in Redshift clusters. This reduces snapshot size and can improve performance.
- Implement cross-account snapshot copying: Instead of just relying on cross-region storage, consider copying snapshots to a separate AWS account. This approach adds a layer of security.
- Monitor snapshot usage with CloudWatch: Set up CloudWatch metrics to track snapshot storage size and usage. Alerts based on snapshot volume or cost thresholds help you adjust snapshot policies proactively.
- Regularly test snapshot recovery: Periodically restore a manual snapshot to a test cluster to ensure data integrity. This practice can reveal issues early and provide confidence in your recovery strategy.
Best Practices for Effectively Managing Amazon Redshift Snapshots
Here are some of the ways you can ensure your Redshift snapshots are properly managed.
1. Use Snapshot Manager Utilities
Snapshot Manager tools such as AWS Backup or third-party orchestration utilities allow for fine-grained control over snapshot scheduling, retention, and compliance enforcement. These tools enable you to configure point-in-time recovery (PITR) objectives and create layered backup strategies that span daily, weekly, and monthly snapshots.
A multi-layered backup plan ensures redundancy and minimizes data loss in case of failure. For instance, you can automate frequent snapshots during business hours and retain daily backups for a week, weekly backups for a month, and monthly snapshots for a year. This provides coverage across various recovery scenarios and aligns with business continuity plans.
These tools often integrate with tagging strategies to associate backups with environments, workloads, or compliance requirements. This allows you to quickly identify, audit, and manage snapshots across multiple Redshift clusters or AWS accounts. Automating these tasks also reduces the likelihood of human error and ensures consistent adherence to backup policies.
2. Validate Snapshot Consistency and Transaction Isolation Awareness
Ensuring snapshot consistency is critical when dealing with transactional or frequently updated datasets. Redshift snapshots are transactionally consistent, but it’s important to take them at safe points—preferably after ETL jobs, batch processes, or schema migrations—to avoid capturing intermediate states.
You should coordinate snapshot creation with your workload’s transaction boundaries. For example, taking a snapshot immediately after a COMMIT ensures the snapshot reflects a complete and valid dataset state. Avoid taking snapshots during long-running transactions or while large updates are in progress, as this can lead to inconsistent recovery points.
Also, be aware of isolation levels in concurrent workloads. Since Redshift uses serializable isolation, multiple queries can read the same consistent snapshot. Still, taking backups without coordinating with transactional workflows can result in snapshots that miss recently committed data or include partially applied changes. Using lock or process completion signals can help trigger backups only when data integrity is assured.
3. AWS Lambda Integration
Integrating AWS Lambda into your Redshift snapshot strategy can simplify the backup process, especially when you need custom automation or need to trigger backups based on various conditions. For example, you can configure Lambda functions to take snapshots after certain processes or events are completed, such as data loading, schema changes, or other important operations.
Lambda also provides the flexibility to integrate snapshots with other AWS services. For example, you could set up Lambda to automatically copy snapshots to other regions or AWS accounts after they’re taken, giving you better control over backup locations and security.
Lambda’s event-driven architecture means that backups can be automatically triggered by specific CloudWatch metrics or alarms, such as storage thresholds being exceeded or certain errors occurring in the Redshift cluster.
Additionally, Lambda can be used to manage snapshot retention policies by triggering snapshot deletions or transfers based on certain timeframes. This flexibility ensures that you can automate everything from snapshot creation to long-term storage management without needing constant manual intervention.
4. Use Retention Policies
Redshift offers you the ability to manage both automated and manual snapshot retention, with automated snapshots being deleted after a set period (up to 35 days). However, for longer-term retention or compliance reasons, manual snapshots offer greater flexibility. By setting up a snapshot retention policy, you can automatically delete older snapshots that are no longer needed, ensuring that storage remains within your budget.
For example, you can define a policy where snapshots older than six months are automatically removed, but critical snapshots (such as those taken before major schema changes) are retained for a longer period.
You can also configure retention policies in combination with compliance requirements. Some industries require data to be retained for years, so creating a retention strategy that aligns with these standards is essential.
For example, you may decide to keep manual snapshots for extended periods in cases where they are crucial for auditing or regulatory purposes. Combining manual snapshots with well-defined retention policies also helps reduce the risk of losing important historical data by ensuring that backups are kept only as long as necessary while managing costs efficiently.
5. Storage Optimization
Redshift snapshots can become large, especially when your cluster contains substantial data. Optimizing the storage of these snapshots is important for reducing costs and improving efficiency. One key method for reducing snapshot size is to regularly clean and vacuum your Redshift cluster. The VACUUM operation reclaims storage by removing unused space and reorganizing data on disk, which in turn reduces the snapshot size.
The ANALYZE operation is also important, as it updates table statistics, allowing the query optimizer to generate more efficient query plans. Running VACUUM and ANALYZE before taking a snapshot ensures that the snapshot captures the most optimized version of your database, minimizing the amount of unnecessary storage consumed.
For long-term storage, Redshift snapshots are typically stored in Amazon S3, where you can reduce costs by using cost-effective storage classes such as S3 Glacier or Glacier Deep Archive. These storage classes are suitable for infrequently accessed snapshots, providing a balance between cost and accessibility. However, if you need to restore snapshots from Glacier, it may take longer compared to standard S3 storage classes, so plan accordingly.
Even When Done Properly, Automated Snapshots Are Not Enough
AWS automated snapshots offer great capabilities that might suit your needs. However, the catch is that you never know when the next recovery will happen, which is where manual snapshots come into play. Maintaining a comprehensive backup of your production environment and running a compliant environment can and needs to only be done by automating manual snapshots. They allow the flexibility and control that you need in order to support your specific custom backup policy and create robust and reliable cloud operations.
How to Fully Automate Your Redshift Manual Snapshots with no surprises
Sure, you can wrangle Redshift snapshots through the AWS Console or CLI—but why bother with that hassle when you can automate the entire workflow? From cross-region copies to policy-based retention, it’s a few clicks and done. In order to define your backup policy and to automate your Redshift snapshots accordingly, you can use an AWS backup solution such as N2W. N2W is a native cloud backup and recovery solution for any Amazon EC2 instance, EBS volume, RDS database, Redshift Cluster, Aurora, DynamoDB or Amazon EFS.
Once you complete the initial setup, there is no need to worry about any failures as if anything goes wrong for any reason, N2W will send alerts and daily summaries. Also AWS SNS can be integrated to send you a notification so you can troubleshoot the error.
Here’s what you can do with N2W:
- ⏱ Set backup frequency for snapshots—hourly, daily, or every five minutes if you want.
- 🌍 Copy snapshots across regions and accounts for high availability and disaster recovery.
- 🧹 Implement lifecycle policies to auto-delete old snapshots and cut storage costs.
- 📊 Monitor usage and get alerts for complete visibility and control.
Want to simplify Redshift snapshot management and sleep better at night?
Try N2W free for 30 days and see how effortless Redshift backup can be.
