I’m working on a project where a large Hyperscale environment needs to be migrated to Azure.
Looking at using either Azure storage or Metallic recovery reserve and/or possibly a mix of the two. There’s short term 30 days as well as LTR 5 to 7 years so will use Hot / Cool tiers (possibly combined storage tiers if Metallic not used)
HS is setup using the default DDB block size 128KB. In an ideal world - one could just setup a secondary copy for the cloud storage (either Metallic or Azure) and kick off the Aux copy to cloud, just let it run to get the data over in to Azure then eventually promote it to primary copy…
As the cloud storage will then be used as a primary copy - ideally, we want to configure it with 512KB DDB block size. Media agents will be setup in Azure as they will eventually become the production MA’s once things get cut over.
Some key questions on the above:
- copying between storage policies with different DDB block sizes – how will this affect overall deduplication savings in Azure if we copy from 128 KB to 512 KB?
- Noted - Performance during Aux copy is affected, that’s been asked and answered quite a few times now I see, however in this case we do have months to get this data across.
- Will data have to be re-hydrated first before being copied over in Azure to new storage policy if DDB block size different? (not been able to find anything in our docs) Assumed data will still be copied across in compress format. Need to understand the network impact too here.
At some point there will be a cut over, however that’s a different conversation since that will be quite complex as not everything will be “switched over in one go” so to speak..
One environment is around 1,5PB of data that needs to be migrated so given the storage size, looking for the best option to keep overall usage to a minimum. (up to 30 days retention is around 180TB of that 1,5PB so it’s mainly LTR)
- Do the aux copy to cloud at 128KB DDB block size and then seal DDB in Azure.
- Once systems across. Create new SP in Azure with default 512 KB and migrated systems start with the new storage policy. From a migration & cutover perspective, this seems the easiest way.
- The option then for some systems to back to HS locally on new 512 KB DDB policy with Aux to Cloud could be used for a short time helping with migration too…
There will be a 10Gb express route enabled for the migration and can be expanded.
Any input on this would be greatly appreciated.
Not sure what the actual numbers will be when you aux from 128 to 512 and what the performance impact will be, but there is an important guideline from Commvault:
512 KB (default) for complete cloud workloads
128 KB for secondary copies that use disk copies as source
So advised is to use 128 KB as you are performing an aux copy from an existing 128 KB based copy.
Only when the workload is fully shifted to cloud I would change it to 512 KB.
Hope this helps.