Skip to main content
Solved

On premise S3 Cloud and DDB data verification


Forum|alt.badge.img+7

Hi,

i have a question about DDB data verification on a Scality ring S3 cloud (on premise).

We have around 800TB of stored data on S3.

Today the ddb verification move on average 1000 TB every month (1.5X as the volume of data stored on S3 scality).

We have set the DDB data verification option to “Quick Verification of Deduplication Database”

it is possible to reduce the number of GET per month by changing the way DDB verification run?

regards,

 

 

Best answer by christophe

Orazan wrote:

@christophe Good afternoon.  I apologize but I do not know what exactly the API used is.  I will see what I can find out.

 

Hi @Orazan , i have the answer for my question with the API use by the quick verification process.

the quick verification perform a HEAD instead of a GET, this reduce a lot the load (data read and network bandwith) on the backend side (scality ring on premise).

We observe on scality ring monitoring a big difference between the two options for Data verification.

View original
Did this answer your question?
If you have a question or comment, please create a topic

5 replies

Forum|alt.badge.img+15
  • Vaulter
  • 630 replies
  • March 16, 2022

Good morning.  There is no setting to change the way that the DDB verification runs.  For cloud settings we generally recommend not to run DDB verification unless needed as the process needs to read many files, which is the reason for the high “GETs”.  By “unless needed” I mean if you start to see errors related to issues with the data.  You can also reduce the frequency that DDB verification is run.


Forum|alt.badge.img+7
  • Author
  • Byte
  • 25 replies
  • March 18, 2022
Orazan wrote:

Good morning.  There is no setting to change the way that the DDB verification runs.  For cloud settings we generally recommend not to run DDB verification unless needed as the process needs to read many files, which is the reason for the high “GETs”.  By “unless needed” I mean if you start to see errors related to issues with the data.  You can also reduce the frequency that DDB verification is run.

Hi @Orazan , Thanks for your reply. 

Do you know what is the API use for a quick verification instead of a full verification (GET ) ?

Regards,


Forum|alt.badge.img+15
  • Vaulter
  • 630 replies
  • April 18, 2022

@christophe Good afternoon.  I apologize but I do not know what exactly the API used is.  I will see what I can find out.

 


Forum|alt.badge.img+7
  • Author
  • Byte
  • 25 replies
  • Answer
  • April 22, 2022
Orazan wrote:

@christophe Good afternoon.  I apologize but I do not know what exactly the API used is.  I will see what I can find out.

 

Hi @Orazan , i have the answer for my question with the API use by the quick verification process.

the quick verification perform a HEAD instead of a GET, this reduce a lot the load (data read and network bandwith) on the backend side (scality ring on premise).

We observe on scality ring monitoring a big difference between the two options for Data verification.


Forum|alt.badge.img+15
  • Vaulter
  • 630 replies
  • April 22, 2022

@christophe Thank you very much for updating with the answer to your question.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings